Two new AI systems beat humans in complex games

Two new articles from AI giants DeepMind and Meta describe how AI systems are winning victories against human players in complex games involving deception, negotiation and cooperation.

Why is this important: Machine competitors have struggled with games where information is incomplete or hidden from players – similar to the intentions of humans in everyday life and interactions.

Driving the news: DeepMind researchers describe a new autonomous agent called “DeepNash” who learned to play the game Stratego in an article published today in Science.

• Stratego is played between two people who each move 40 pieces with different ranks – which cannot be seen by their opponent – in an effort to capture the other player’s flag.
• DeepNash couldn’t play Stratego looking for every possible scenario because there is an “astronomical” number, writes the DeepMind team – far more than chess, Go and poker, which AI systems have beaten.

How they did: The DeepMind team combined an algorithm for learning the game through self-play and another that directs this learning towards an optimal strategy.

• DeepNash learned the game from scratch by playing around 5.5 billion games against itself over four months.
• The AI ​​Agent beat other existing Stratego bots, which play at an amateur level, more than 97% of the time, they report. He won 84% of the time against expert human players on an online gambling platform, sometimes by bluffing and cheating.
• DeepNash can “handle huge amounts of uncertainty in the form of imperfect information, more than was previously possible,” said co-author DeepMind researchers Julien Perolat and Karl Tuyls in an email. main points of the article.

Meta-researchers last week described an AI system called “Cicero” which they claim can play the game Diplomacy on a human level.

• In Diplomacy, up to seven players negotiate, deceive, and build alliances in an attempt to gain control of territories on a map.
• “Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players’ beliefs and intentions from his conversations and generating dialogue in pursuit of his plans,” they wrote in Science.
• In 40 games of a blitz version of Diplomacy where the time for each move is limited to five minutes, Cicero scored more than double the average score of human players he faced on a gaming rig.

To note : Some experts say the AI ​​systems that can play these games raise concerns about the machines’ ability to deceive.

• Cicero “passed as a human player in 40 matches” with 82 unique players, Meta researchers reported.
• They also said they were able to control AI dialogue to be “largely honest and helpful”.

The big picture: Experts debate the extent to which mastering games will help develop intelligent machines that can navigate the world of humans.

• Some argue that their rules are specific and that winning on the board does not easily extend to a range of real-world issues.
• But others say some of the skills required to win strategy games could lead to real-world applications.

Debate is “a wrong question,” says Tuomas Sandholm, a professor at Carnegie Mellon University who has studied game theory for three decades.

• “When it comes to planning against opponents, games are not only important, they are necessary,” he says, adding that they can provide insights into negotiation and reasoning for business, finance and defense, which the two companies he founded and lead, Strategy Robot and Strategic Machines, focus on.

Yes, but: A board game is a “highly controlled and constrained environment,” says Luca Weihs, a researcher at the Allen Institute for AI who works on how systems can be physically incorporated to control a robot or vehicle.

• But in the real world, an AI system might see a human do something it’s never seen before – and have to reason the person’s goal through common sense.
• Hidden information and intentions abound in everyday life, even in seemingly simple tasks like helping a partner load the dishwasher or driving a car down a street alongside other drivers.
• “We’re constantly working with missing information about how humans function and very intuitively filling in the gaps.”

People can also adapt to changes in game rules or board structure, says Brenden Lake, who co-directs the Minds, Brains, and Machines initiative at New York University.

• “Many high-level systems would be completely destroyed by a change in the rules, or the size and shape of the game board, if they didn’t have the ability to retrain.”

What to watch: Cicero relies on a more classic approach to AI that involves training it on a corpus of human games and other bespoke data, giving it innate knowledge, writes researcher Gary Marcus.

• This is in contrast to systems like DeepNash, which is trained entirely from scratch through self-play. Cicero also uses autoplay data.
• Marcus has long argued that the deep learning approach, which powers chatbots, virtual assistants and self-driving capabilities and has been a focus of researchers, is limited.
• It can “ultimately prove even more valuable if integrated into highly structured systems,” he writes.