In recent years, OpenAI, a startup with the mission to ensure that "artificial general intelligence benefits all humanity", has developed a machine-learning-driven bot to play Dota 2, the largest game in the universe. Starting with a very simplified version of the full game, the bot has been developed over the years by playing millions of games against themselves, not only learning how to play the five-to-five team, but also how to win, consistent.
We were able to follow the development of the bot over a number of show competitions, in which everyone used a more complete version of a game and more experienced human opponents. This culminated in what is expected to be the last show match in the weekend, when OpenAI Five was released in a best-of-three match against OG, the team that won the biggest competition in all of the esports last year, The International.
OpenAI is subject to a number of handicaps in the name of keeping things interesting. Each of the five AI players has an identical version of the bot software, with no communication between them: they are five independent players who happen to be very similar, but have no direct way to coordinate their actions. The response time of OpenAI is artificially delayed to ensure that the game is not simply a showcase of superhuman reflexes. And the bot still does not use the full version of the game: there is only a limited selection of heroes available and items that create manageable henchmen or illusions are forbidden because it is thought that the bot can micromanage its minions better than any human would that can.
The games can be viewed here. The first game even looked up to about 19 minutes. The people had a small gold advantage, but the bots had better territorial control. The bots got ahead in a team fight, killing three human players while losing only one. The game was still looking forward, but the bots disagreed: they announced they had a 95 percent chance of winning and made immediate use of their number advantage to do serious damage when making this statement to bring to the human base. This further strengthened their territorial control and also gave them an important golden lead.
This put the people in the background, and while they managed to pull the game off for another 20 minutes, they couldn't overcome the lead of the bots, giving OpenAI a 1-0 advantage.
In the second game things were not even close; the bots took an early lead and violated the human base within 15 minutes. They took the win five minutes later.
Overall, it was a dominant performance from OpenAI: a 2-0 win against an established human team accustomed to playing with each other at the highest level the game has to offer. This performance was by far the strongest of OpenAI over the years.
The coordination of the bots is creepy: although they cannot communicate, all five computer-controlled players think the same way. If someone thinks it's a good chance to attack a human player, the other four will think the same and participate in the attack. This gives the appearance of good team combat coordination – coordination with precision and accuracy that human teams cannot match.
But OpenAI looks addictive. It has clear, or surprising, weaknesses – it's not great at scoring the latest hits, the deadly blows on computer-controlled units used to collect in-game gold. This gives people the chance to gain an early gold advantage. The bots also struggled to prevent invisibility on the human side. They also seemed to adapt poorly to certain spells of some heroes, particularly Earthshaker's Fissure, a spell that temporarily creates an impassable barrier on the map. People were effective in using this to catch bone players and restrict their movements, and this seemed to confuse OpenAI.
The behavior of the bots is also an object lesson in the big gap between this kind of machine-learning system and a complete general artificial intelligence. Although AI Five is clearly effective in winning games, it also clearly does not know how to play Dota 2. Human players of the game use a technique called "pulling" to transmit the flow of computer-controlled subordinates from their side (also called engerds) Dota 2) as a way to deny the enemy team both gold and experience. Human players can recognize that this has happened because they do not show up when it belongs. Human players have a mental model of the entire game, an understanding of the rules, and can therefore recognize that something is wrong; they can reason about where the creep must have gone and disrupted the attraction. The computer, on the other hand, runs aimlessly in this scenario.
In the millions of games played against itself, OpenAI never seems to have picked up the technique of pulling, so it has never learned to play against it. So when a human team starts pulling, the bot doesn't recognize the situation and doesn't really know what to do. It cannot reason about what the game should be like, and it cannot speculate why the game behaves in an unexpected way. The only thing the bot can do is look for patterns it recognizes and choose the action that most likely produces the best results; give it a pattern that it is can not and deteriorate its performance.
Up to now, the OpenAI bot is limited; certain pros and streamers have been given access to play against, and it is also available to play against some live events. But for a few days, that is changing: Dota 2 players can sign up here to play against the bot or with it for a period of three days. Unfortunately, this public period does not seem to result in a new and improved bot: defeating a top team is the goal that OpenAI has set for his bot, and that completes the experiment.