Meta’s board game-playing AI can pass as a human in game negotiations

An AI developed by researchers at Meta convincingly negotiated with humans while playing an online version of the board game Diplomacy. It was also ranked in the top 10 per cent of players

By Jeremy Hsu

An AI that is an expert player of the first world war strategy game Diplomacy can now participate in game conversations and negotiations without most human players realising they are talking to a machine.

The game of Diplomacy involves seven players competing for control over European territories. But there is also room for cooperation as players negotiate temporary alliances in coordinating movements and attacks.

Researchers at Meta, the parent company to Facebook, built an AI named Cicero that learned to play Diplomacy through trial and error and by partially mimicking what human players typically do. This approach previously allowed the AI to beat expert human players in a simplified version of the game that didn’t involve negotiation

The team has now added a language component to Cicero so that it is capable of interpreting and generating messages. The researchers started out with a pre-trained AI system capable of analysing language and further trained it on 12.9 million human messages from more than 40,000 games of Diplomacy. They also developed methods for filtering out nonsense from Cicero’s messages.

Such language capability allowed Cicero to understand human intent from messages and use that information to inform its actions, says Emily Dinan at Meta. At the same time, Cicero used its strategic reasoning capability to generate messages linked to its goals in the game.

The researchers entered Cicero anonymously in 40 games of Diplomacy hosted by an online league of human players. The AI sent 5277 messages to human players over the course of 72 hours of gameplay – and almost nobody wised up to the fact that they were communicating with an AI. Just one player voiced suspicion in a post-game chat that one of the AI accounts might have been a bot. Cicero ranked in the top 10 per cent of players across the games.

“What really impresses me is how the strategic reasoning and language models come together,” says Julian Togelius at New York University.

One ongoing challenge for language AI systems is that they often replicate biases. For example, Cicero had a tendency to assume players were male. This was further complicated by human players often using aggressive or militaristic language specific to Diplomacy. The Meta researchers developed language filters that caught most but not all toxic language generated by Cicero, though they did not share specific examples where things went wrong.

Cicero’s success came in an online version of Diplomacy where each negotiation turn is limited to 5 minutes. “Even though Cicero yields very promising results against the human players it faced, the short turn times of this game variant give Cicero an unfair advantage,” says Georgios Yannakakis at the University of Malta.

It was still no small feat for the AI to pass as human while outplaying most human opponents. Cicero generally took an honest and helpful approach to negotiations where many Diplomacy players use deception and lies – but that didn’t mean the AI was completely naive.

“There are also situations where it may withhold strategically valuable information to help it achieve its objectives,” says Dinan.

Journal reference: Science, DOI: 10.1126/science.ade9097