Can we have it in the new Microsoft Flight Simulator?

An artificial intelligence system trounced a human fighter pilot in a virtual dogfight – repeatedly.

In a competition run by the Defense Advanced Research Project Agency, Heron Systems’ AI was able to outperform a human fighter pilot sitting in a cockpit simulator and wearing a virtual reality helmet, five times in a row.

The system also defeated seven AI-based competitors created by rival teams. Each entrant had to run through multiple scenarios developed by John Hopkins Applied Physics Laboratory, including one involving a cruise missile.

Calling in reinforcements

There were considerable limitations to the AlphaDogfight match. Both pilots of the virtual F-16s could only use front-facing cannons to keep things simple; program manager Lieutenant Colonel Dan Javorsek admitted that limitations in the simulation could mean that the AI was inadvertently abusing "some weird digital artefact that we weren't thinking about or considering," that would not translate into the real world.

Still, the victory marks a significant milestone in the history of AI and military engagement.

Heron and the other competitors built their systems in a year; both Heron and Lockheed Martin embraced deep reinforcement learning – where the AI system simulates a task again and again, until it somewhat understands what is happening.

“We didn’t try to combine any kind of expert systems," Ben Bell, Heron’s lead engineer for machine learning, said in a Q&A following the victory. "Our advantage was only using reinforcement learning so we tried to create an action scheme where the neural network could control the plane in a way that was smooth, which you didn’t see our competitors doing, and at a high enough rate.”

The second most important part of the system, Bell said, was the training architecture: “We started off early with a league of agents. We want to create multiple different agents that fly in different patterns, with different reward structures and different neural network architecture. So the final agent we used has trained against 102 other totally unique agents, and that made us robust enough to beat any opponent, even the human pilot.”

The winning agent had simulated at least four billion flights and fights, accumulating the equivalent of "12 years of experiences."

One area that it did not learn from, was the fights themselves. As part of the contest's rules, the system was not allowed to analyze each of the rounds – meaning that by the fifth round the anonymous human pilot was able to change his tactics, and last longer.

© DARPA

Timothy Grayson, director of the Strategic Technology Office at DARPA, noted that in the nearer-term, what this match foreshadows is not a completely AI-powered fighter jet, but "human-machine symbiosis."

He added: "Let’s think about the human sitting in the cockpit, being flown by one of these AI algorithms as truly being one weapon system, where the human is focusing on what the human does best and the AI is doing what the AI does best."

The vision is all part of DARPA's Air Combat Evolution, or ACE, program, which “seeks to automate air-to-air combat and build human trust in AI as a step toward improved human-machine teaming.”

Separately, the US Air Force runs the Skyborg project, which aims to deploy XQ-58 Valkyrie drones alongside F-35 and F-15EX fighter jets as 'AI wingmen.'

The drones would fly next to the jets, watching and learning from them, while also offering the ability for remote control.

It is not clear if the project is related to DARPA's work, and public documents make no mention of the Joint Artificial Intelligence Center, which was set up to help coordinate the numerous AI and big data projects that dot the sprawling military-industrial complex.

DARPA admitted there was a large PR element to the AlphaDogfight competition – but the intended audience it is trying to win over is the military itself.

"The fighter pilot community is very reluctant towards any sort of change," Javorsek told Breaking Defense. "There’s almost an institutional requirement to not accept change. When it comes to autonomy, this has been avoided almost like the plague.”

DARPA Deputy Director David Honey said that more than 15,000 people signed up to watch the finale, many of them currently in the service.

The entire five-hour event is available to view on YouTube: