The Air Force Research Lab is building a video game for AI wargaming

And it sounds like something I would play

The US Air Force is developing a video game, and is looking for researchers that could use it to help train AI-based warfighting systems.

The Stratagem Wargame, which has not been previously reported, is under development by the Air Force Research Lab in Rome, New York.

The game itself

Procurement documents provide some insight into the government-designed video game that will likely never see a public release.

“It is an operational-level, hybrid turn-based and real-time strategy game played on a hex-based grid,” a broad agency announcement Q&A document states.

“It is meant to incorporate multiple domains, starting with land and air.  It is heavily inspired by concepts from Avalon Hill’s tabletop wargame series, most notably their Stalingrad and Midway titles.”

The map will include different land types, including wooded, clear, city, village, water, river, marsh, road, hills, mountains, rubble, factory, water edge. Future versions will have a greater maritime focus.

The model layer for the game is developed in pure C#, while the visualization and controller layers for human players are developed in Unity3D - although one assumes the game puts less emphasis on visual effects than commercial titles.

Stratagem Wargame will have turns, much like tabletop games. Each turn will be split into three phases: planning, execution, and after action.  

“Players give their units orders in the planning phase. The units execute those orders in the execution phase (movement and combat ensues). Players may inspect the results of the execution phase in the after action phase.” Turns constitute a single day, with matches lasting up to 100 days.

At first, the game will support just two players, but might be expanded to include a third.

Different troop types will exist, with different attack and defense capabilities. They will also suffer from variables such as fatigue, AFRL states: “If a unit is highly fatigued through sustained movement, highly disorganized through numerous combat losses, and undersupplied, it will certainly go last during the execution turn phase in the game.”

Don’t expect individual troops to be detailed entities: to save on processing power and development time, “the Government is looking at low fidelity, effects-based modeling for units in the wargame.  

"The Government is abstracting away the tactical elements of combat, like utilizing certain weapons for certain troops. Instead, the Government is looking more at the effect, the combat factor, of applying a certain unit in the game. For example, the Government may say a certain unit may have an attack factor of 3.2 and a defense factor of 2.4, whereas another unit may have an attack factor of 4.0 and a defense factor of 3.7."

Weather is currently a "lower priority feature," but will impact ground and cloud conditions across the entire map - which can range from 50 square miles to 500.

This, along with other features, such as more complex supply chain elements, may be upgraded and improved "in the coming months."

There are other game elements that are distinctly unrealistic. Currently, communication between troops is instant and all pervasive. Every troop knows the location of every other troop, and there are no communication limitations like those experienced on a real battlefield. 

Players will be able to see the entire map, without the 'fog of war' commonly found in video games, which limits vision unless in close proximity.

"Future, more advanced versions of the wargame may contain limited Fog of War elements if there is interest from operational partners for these features.  It would be possible to apply fuzziness to some of the intelligence gained on enemy units, where lower degrees of intelligence gathering success would result in more uncertain values."

Equally, complex physics is not modeled - acceleration, rotation, weapons models, and sensors are not simulated. Some of this could be revisited in the future, AFRL said, but the focus is on keeping the game lightweight.

The aim is for the game to be simple enough so that a single server could run thousands of instances at once, training an AI model in a match after match.

Enter AI

Over the next three years, AFRL is soliciting white papers that describe the methods to achieve its game-making goals.

At the moment, the lab is only accepting proposals for Focus Area 1, which aims "to explore the use of new and existing algorithms and learning methods applicable to developing strategies."

A separate procurement document notes: "The end goal is to develop a new state-of-the-art approach for matching and then exceeding expert human gameplay in advanced wargames.”

Submissions can include approaches based on existing commercial video games, such as StarCraft II.

No date has been set for Focus Area 2, which seeks to "develop a capability where a machine can learn to perform complex planning and game play actions via the use of human and AI planners/operators as teachers or opponents."

Finally, there's Focus Area 3 in which the applicants will have to take their existing AI systems, designed to play games like StarCraft, and apply them to the wargame. ”Learning from rich data sources will then become possible and allow for AFRL’s blue and red agents to produce more robust strategies.”

Successful applicants will then have to transfer their AI models from the Stratagem Wargame to other tools and simulations, such as AFSIM and AWSIM, the wargame modeling platforms already used by the DoD.

By the end of the program, the objective is "to have an AI capability that can compete at, or above, the level of capability of seasoned veteran planners in realistic wargame scenarios (that have experience using the wargame tool)."

More than just a game

Video games offer a scaled down model of the real world that can be infinitely generated, allowing for artificial intelligence systems to test themselves again and again, slowly improving - without causing physical havoc.

The best-known and most advanced attempt to train an AI on a video game is AlphaStar, a project by Google's DeepMind that designed a self-learning model which got to a point where it was beating the world’s top StarCraft II players.

AFRL’s announcement makes multiple references to AlphaStar, but notes that DeepMind had the luxury of access to copious amounts of player data to train its model on - after all, StarCraft is one of the world’s most popular video games.

Participants in the ARFL contract will not be so lucky, with Stratagem likely remaining a closely-guarded secret. “Therefore, if machine learning or other data intensive methods are to be used to develop strategy to play the wargame, methods of creating synthetic data, using forms of self-play, or methods of few-shot learning (if possible) may be necessary.”

Outside of the AFRL, other arms of the US military-industrial complex are interested in the combination of video games and AI for improving military capabilities. 

As exclusively detailed by AI Business last week, DARPA is planning to train AI systems on commercial, open-world video games, including StarCraft II. The Gamebreaker project is looking for AI systems that can destabilize simulations, coming up with ways to change the game entirely.

The Department of Defense's interest in video games dates back even further: in 2002, the military published the first installment of 'America's Army,' a first-person shooter designed ostensibly to help train soldiers, but also used as a recruiting tool.

Numerous sequels, spin offs, and even an arcade version has been developed. America's Army 5 was revealed to be in the works in 2018, but is yet to be released.

The military was also involved in the production of Full Spectrum Warrior, a 2004 real-time tactics video game. While it was ultimately developed by Pandemic Studios and published by THQ (both now defunct), the original version was funded by the US Army. 

Two versions were made, one military and one commercial, as the Japanese-owned PlayStation platform was not allowed to be used for a military purpose.

Such restrictions did not extend to the use of PlayStation hardware, however. In 2010, realizing that the powerful PlayStation 3 - with its proprietary Cell chip and Linux capability - was significantly cheaper than a supercomputer, the AFRL acquired 1,760 PS3s.

The Air Force's "Condor Cluster" became the 33rd most powerful supercomputer in the world and ‘the fastest interactive computer in the US Department of Defense.’

In its place, AFRL now operates four conventional supercomputers, with the most powerful clocking in at 4.87 peak petaflops of performance.

It is also housing the world's largest neuromorphic digital synaptic supercomputer, using IBM’s TrueNorth technology. An unclassified presentation from last year revealed that the AFRL operates "two quantum labs and two neuromorphic/electronic labs," and references potential quantum computing partnerships with IBM, PsiQuantum, IonQ, and others.