And against AI. What shooters can you play with bots now? Game against a neural network

A bot created on the basis of a neural network defeated a famous Dota 2 player on the main stage tournament The International (TI) 2017. The main competitions in Dota take place between teams of 5 vs 5 people, but there is also an individual competition. Open AI created an AI that trained itself by playing against itself, 1 on 1. On stage, the bot beat a popular player 2-0 Daniil Dendi Ishutin from NaVi.

Before this, behind the scenes of the competition, the bot fought with SumaiL - best player 1 on 1 - and with Arteezy - the best player in the world, his MMR is more than 10,000. Their bot also won “dry” - 2-0.

Fun fact: after the tournament, everyone was invited to try their hand at the winning bot for rewards. By the evening of the same day, the bot was defeated by players from the Internet with an MMR of about 8000. It took several attempts to identify weakness. However, the bot won on the tournament stage.

Why is this an important event?

The tournament is organized by Valve, the game's developer, with a prize pool of almost $25 million. This largest eSports competition sets a new record every year for the number of views and prize pool. International is watched online by half a million viewers on Twitch, several hundred thousand more through the game client, and millions more in recordings.

Level 2. How does it work?

Let's go down a level deeper (or higher, depending on how your imagination works). Let's find out!

The developers decided to approach the problem from scratch. They realized that if they tried to describe the number of rules in open game with thousands of variables, it will take several months. Instead, they set up a bot, brought it into the game and gave it complete freedom of action.

What evolution looks like:

1. The bot played against itself, the same bot. They exchanged experiences.
2. For the first few games, the bot stood still and did not move, after which it began to perform random actions or movements.
3. After thousands of attempts, the AI ​​came to the conclusion that you need to go to the center of the map and fight the enemy in order to win.
4. And after several weeks of continuous training, the bot began to play at the level of first-class players.

During training, the AI ​​learned:

• make the final blow - this brings bonuses in Dota;
• provoke small enemy creatures to attack;
• slow down your creatures to better group them;
• keep the enemy at a distance from your creatures, depriving the enemy of money and experience;
• cancel ability animations to avoid taking damage;
• bluff canceling ability animations, which leads to a defensive reaction from the enemy;
• adapt to new items that the enemy buys;
• pursue the enemy by predicting his possible location in the fog of war.

To visualize the process of training a neural network, look at the video:

How is a neural network different from playing against a computer?

Game against the computer

The main difference between the Open AI bot and the enemies built into the game is the technology on which they are based. The game's built-in enemies are based on a decision tree. Solutions are found by searching through combinations of input parameters and solving optimization problems. Such decision trees include a fairly large number of parameters, so the player will feel like he is playing against a living or adapting opponent.

Nevertheless, all possible variations in the behavior of such opponents are predetermined by the creators of the game. Built-in bots skillfully imitate intelligence, but do not possess it. Creating such artificial opponents is comparatively easier and faster for developers.

Game against a neural network

Neural networks take a different approach. The network that Open AI used is based on a principle that is similar to evolution. By performing random actions, the network creates various solutions. Solutions are evaluated by a fitness function.

Successful solutions connect with each other, mutate and create new generations of solutions. New generations are compared with their “parents” to understand whether progress has appeared. If there is no progress, the “parent” solutions are crossed in new proportions. The process is similar to the evolution of organisms in nature. Such an algorithm does not always lead to an ideal solution, but it will come very close after going through generations of errors.

In the video, the chief engineer of the project talks about the process of training the neural network when preparing the bot for the tournament:

A neural network-based bot requires significantly more resources than a classic decision tree. Such neural networks are used to design jet engines and find cures for Alzheimer's disease. Open AI used the computing power of the Microsoft Azure cloud server, since Microsoft is one of the founders of Open AI.

Musk's tweet after the bot's victory in the tournament:

The entire process of training neural networks regarding the fitness function can be seen in a video about the game Mario. Using the method of neuroevolution of complementary topologies, or simply evolution by trial and error, the author of the video created a neural network that passes a level in Mario.

At the beginning of training, such an AI does not even know how to walk, after 34 generations and several days of training, the AI ​​was able to complete the level without dying:

Is winning in Dota really that important? What do the experts say?

I went through programming forums, reddit, and communities of neural network enthusiasts. The opinions of the participants were divided: there was both enthusiasm and strong criticism.

Admiration

Translation: The bot wins by slowing down its creatures with the character's body. This is awesome.

Translation: As I understand it, the test was to see how the bot learns to play on its own, without given rules and frameworks. He learned and beat the best. Don't get me wrong, this is great progress.

Criticism

Translation: A bot that wins against a professional in Dota is comparable to a robot that shoots balls into a basketball basket from a standing position. This is not real basketball and not real Dota.

Translation: It sounds much less impressive when you realize that the bot is, by definition, equipped with tools that allow it to have superhuman reactions and accurate knowledge of damage in numbers. This allows you to accurately calculate the damage and deliver the final blow. It will be really interesting if they can teach the bot to play macro strategies that are not available in a 1v1 game. If they can win evenly against a team of 5 people, that will be an achievement.

There were also some who were completely doubtful:

What is the conspiracy theory?

In connection with the release of the API from Blizzard, a conspiracy theory arose on the network. There's a reason tech giants suddenly focused on computer games.

Seconds after the victory of the Open AI bot, Elon Musk tweeted that the bot organization, initiated by Elon himself, won a much more complex game than chess or Go:

The tweet is likely a boast to IBM and Google, who have mastered "simpler" games. Don't forget that Musk and his colleagues have invested more than a billion dollars in the non-profit research organization Open AI.

At a minimum, they will be able to attract the best specialists, who will subsequently be offered jobs at Tesla or other sponsoring companies. On forums and on Twitter, Musk was accused of clickbait and exaggerating the merits of Open AI.

Developers of bots for Dota 2 were dissatisfied

Valve did not invite programmers to the tournament who worked on creating third-party solutions for Dota 2. There are several programs based on decision trees and other technologies that are capable of playing even in 5v5 mode. However, an early prototype for the game 1v5 was demonstrated on stage. 1 by Open AI. Valve did not comment on this claim.

In conclusion, I want to say: “I, for one, welcome our new robot overlords.” As for me, I welcome our future robot guards. And I regularly write about them in our Telegram channel Robotics Channel. Join us!

And so, the long-awaited Heart of the Swarm beta test. We put aside vague interviews, screenshots and battle reports, and rush to check our mailbox for the treasured beta key...

Alas.
The vast majority of average users will be disappointed in these early days of beta. Blizzard has officially announced that the first beta testers will include a select group of pro gamers, members of the press, and winners of Torgovy Row competitions.

But you and I will not put up with such injustice, right?

And for this reason, especially for you, I will try to tell you in as much detail as possible about how to play, if not on the official Battle.Net, so easy at home with a computer. And there, perhaps, Star Friend with his pirate LAN & Hamachi will catch up...

There is such a wonderful program: SC2 All in 1. Which allows you to play StarCraft 2 Wings of Liberty not having license key and/or Internet access. True, only against artificial intelligence(AI). Which, however, at high difficulty levels plays not much weaker than players from the “Iron Leagues” (below diamond, that is). And also view replays.

Latest version at the time of writing 11.2.11 beta 8 already has support Heart of the Swarm v2.0.0.23077 beta .

Well, let's get started.

1. Download a pack of cards (if you are also going to play WoL offline on a computer)
2. The program may require .NET Framework 4 for correct operation

Unpack the archive with SC2Allin1 to any folder and run SC2ALLin1.exe. (By the way, if some particularly paranoid antiviruses send you to the “King Trojan”, then first calm down the alarmists - there’s nothing wrong with that.)
The following window appears before us:

Despite the abundance of options, you won’t be able to launch anything, because "No map is selected!". Those. First you need to download the maps you are going to play on.

Go to the tab Maps .

Here we are offered a choice of four buttons:

• Add Maps from Folder- add maps from the selected folder. This is if you have previously downloaded the maps. From the above link, for example.
• Select Map- select a card and go to the game settings (Game tab)
• Map Sites- launches the browser and moves us to a page with links to useful online resources, where these same maps can be downloaded.
• Add Maps from Server- don’t bother searching for maps on the Internet, but simply download them from the server.

Click Add Maps from Server . In a new window you will find a list of all supported cards SC2ALLin1. We are interested in those cards that are marked (hots). Check them off and click Download and Install Selected Maps . Downloaded maps can be found in the folder "My Documents\StarCraft II\Maps"

Click the invisible cross in the upper right corner and close the map loading window.

Now we have the names of the maps we downloaded in the list. The number on the right indicates the number of players for whom it is designed. On the left side of the window there is a preview, information about the creators and a commentary on the map. The numbers on the diagram indicate the locations of the starting positions of the players.

Click Select Map and return to the very first window of the tab Game .

Now let's move on to the game settings.

The leftmost, nameless list is responsible for choosing a race:

1. Random
2. Terran
3. Zerg
4. Protoss.

Player Type: set the player type.

1. Human- player, i.e. You.
2. Computer- your silicon rival
3. Referee, Spectator- we are not interested, because we're going to play in splendid isolation.
4. Closed- the slot remains empty.

The game requires at least one computer opponent.

Difficulty: set the difficulty level. The level of intelligence of your computer opponent will directly depend on this option. Options:

1. Very Easy- very easy
2. Easy- easily
3. Medium- average
4. Hard- difficult
5. Very Hard- very hard
6. Insaine- computer cheater.

Basically, the set of options and their difficulties depend on the version of AI you are using. This assembly includes a brain dated May 10, 2012. release dates. For Humans, the ability to choose is prudently blocked.

In the near future I will update the article by explaining in more detail about setting up AI.

Color: allows you to choose a color for yourself and your opponents. The number of options here is much more extensive than in most custom maps. The following are available to choose from:

1. White
2. Blue
3. Teal
4. Purple
5. Yellow
6. Orange
7. Green
8. Light Pink
9. Violet
10. Light Gray
11. Dark Green
12. Brown, Light
13. Green
14. Dark Gray
15. Pink

Just like a children's set of 16 markers. Plus more Random(random color).

Location: selection of starting position from 01 to 08 or Random. You start at the specified location, according to your choice.

Health: unit health reserve of the selected player. From 100% to 5%, from the original. If you want to kill everyone as slaves or give your opponent a good head start, you can play with these values.

On the right side there are two more options:

Game Speed: game speed. is by default Faster. If you want to see a battle of slowpoke players, you can move the slider to the far left position. (Slower)

Game Type: game type.

1. Team vs Team- team-to-team
2. Free For All- aka FFA, everyone against everyone. Installed by default.
3. Use Map Settings- the mode is set by the map settings.
4. Random War- something random. :)

If selected Team vs Team, additional items will appear in the settings:

1. Team- allows you to select a player’s affiliation to a specific team.
2. Team Shares: general review ( Vision), unit control ( Control) and resources ( Spendings). Or all of this at once.

On this moment modes work correctly Free For All And Team vs Team. Check mark Custom AI is responsible for using the original Blizzard script (no checkbox), or a third-party script (checkbox). By default, a third-party script from the folder is used \SC2Allin1\ai

Next: go to the tab Options

In chapter Others enter your nickname (if you don’t want to be just a Player). NECESSARILY check the box and indicate the path to the folder with HotS Beta. (x:\Games\StarCraft II 2012 Beta- For example). Otherwise, the program will by default launch the one installed on your computer. Wings of Liberty. (After specifying the path and checking the box, it is better to restart SC2AllIn1, so that the path is preserved).

You can also play around with the numbers in the section Cheat, giving yourself free resources.

This completes the settings.

All you have to do is click Launch and voila!...

Heart of the Swarm Beta at your complete offline disposal! Have a good game!

A small FAQ on popular problems and their solutions.

Q: When you try to start the game or resume an interrupted installation, an error appears: "Falled to read required file. Please, close all other applications, temporarily deactivate your anti-virus software, and try again"

A: Delete the folder C:\ProgramData\Battle.net . If it doesn't help, launch the game via SC2Switcher.exe , located in the folder StarCraft II 2012 Beta\Support .

Q: When I try to start it gives an error: "Unable to validate SC2 License"

A: For some reason, the game thinks that you have a non-licensed version. There are two treatment options:

Q: After clicking the Launch button and starting the game, it gives an error: "Unable to open map"

A: Possible solutions:

1. Make sure you are running the game in mode Free For All or Team VS Team.
2. Check that you have downloaded and launched that map - it has (hots) in its name.
3. Make sure you have specified the correct path to the installation location HotS Beta and checked the box Use Custom Game Installation Path.
4. There is no Cyrillic alphabet in the installation paths.

Q: Error messages appear in the game.

A: Just uncheck the left and right boxes:

And most importantly, remember that despite any private distribution of keys -

On October 31, 2017, a duel took place between professional StarCraft player Song Byung-gu and four different AIs. The man won each of the battles. One of the bots was developed by the Facebook FAIR laboratory, the rest were developed by programmers from Australia, Norway and South Korea.

Human vs AI

The competition took place at Sejong University in Seoul, South Korea, where annual StarCraft AI competitions have been held since 2010. In previous years, matches took place only between bots and were organized by the Institute of Electrical and Electronics Engineers (IEEE).

Although the matches in Seoul did not attract as much attention as those between AlphaGo and Go professionals, they are still of great importance in the development of AI. The research community considers StarCraft especially difficult game for bots. Following AlphaGo's success in Go and some AI advances in chess, scientists' attention turned to the idea of ​​teaching AI real-time strategy games such as StarCraft.

Unlike the same Go, in which constant control of everything is possible playing field, due to which players can develop a game strategy at the first step, StarCraft requires using memory and developing tactics on the go within a limited and virtual game world. As a result, StarCraft becomes an effective tool for helping AI take a step forward in its development.

Past battles are just the beginning

Numerous professional StarCraft players have said they are willing to play live matches against bots, similar to AlphaGo matches. The leaders of the DeepMind project have already agreed to organize similar matches in the future.

Returning to the past matches, it is worth noting that the total time spent by Song for 4 matches was only 27 minutes. The longest match lasted 10.5 minutes, the shortest was only 4.5 minutes. This was despite the fact that bots could move their units much faster than humans, as well as manage multiple tasks at the same time.

The Norwegian bot, for example, at one point in time reached a speed of 19 thousand actions per minute. This despite the fact that professional players cannot perform more than several hundred actions per minute.

The winning man noted in the post-match interview:

Professional players begin battle only when they have an army and unit management skills sufficient to win. The bots, on the other hand, tried to save their units without making any bold decisions. However, the way they fought off my attacks was sometimes simply stunning.

Kim Kyung-jong, a professor of computer technology at Sejong University, said the bots' actions were limited, in part, due to a lack of available training data. He also added that AlphaGo was able to defeat Go champions only by studying a large number of matches played by human professionals.

Development perspective

Note that Google has long wanted to train AI to play StarCraft. Also in this pursuit are Facebook and Alibaba. In August 2017, DeepMind and StarCraft developer Blizzard released a long-awaited set of AI development tools compatible with the version of StarCraft II most popular among professionals.

Experts are already predicting the complete collapse of human dominance in StarCraft. According to Jung Han-min, a professor of computer science at South Korea University of Science and Technology, when AI bots will be equipped with decision-making systems high level, such as AlphaGo, people will no longer be able to defeat them.

Year of manufacture: 2010
Genre:
Developer:
Publisher:
Developer's website: http://us.blizzard.com/en-us/games/sc2/
Interface language: Russian
Platform: PC
System requirements:
Installed game Starcraft 2: Wings Of Liberty version 1.0
Description:
StarCraft II is the continuation of the epic saga about three powerful races: Protoss, Terrans and Zerg.
They will face off again in a new real-time strategy sequel to the legendary StarCraft game.
In this brutal struggle for survival in Space, you will have new combat units at your disposal,
as before, with expanded capabilities.

Agria Valley
"Blistering Sands"
"Burial Grounds"
"Crossfire"
"Desert Oasis"
"Incineration Zone"
"Jungle Basin"
"Junk Yard"
"Novice Blistering Sands"
"Novice Desert Oasis"
"Novice Steps of War"
"Scrap Station"
"Steps of War"
"World Ship"
"Xel Naga Caverns"
"Elysium"
"Arid Wastes"
"Debris Field"
"Delta Quadrant"
"Discord IV"
"High Orbit"
"Kulas Ravine"
"Lost Temple"
"Metalopolis"
"Month Ridge"
"New Antioch"
"Nightmare"
"Novice Discord IV"
"Novice Kulas Ravine"
"Novice Metalopolis"
"Novice Monlyth Ridge"
"Novice Terminus"
"Novice Twilight Fortress"
"Red Stone Gulch"
"Sacred Ground"
"Scorched Haven"
"Shakuras Plateau"
"Tarsonis Assault"
"Terminus"
"Twilight Fortress"
"War Zone"
"Arakan Citadel"
"Colony 426"
"Dig Site"
"Dirt Side"
"Frontier"
"Moonsoon"
"Quicksand"
"Tectonic Rift"
"The Bio Lab"
"Typhon"
"Ulaan Deeps"
"Abyss"
"Extinction"
"Forbidden Planet"
"High Ground"
"Lava Flow"
"Megaton"
"Outpost"
"Primeval"
"Sand Canyon"
"Tempest"
"Toxic Slums"

Copy maps to My Documents folder StarCraft IIMaps
- If the "Maps" folder is missing, then you need to create it.
- Go to the game folder and run the file "StarCraft II Editor" or "SC2Editor"
- In the program, select the map you want to play on, then click on the “Map” tab at the top,
- Select the item “Player Options” and set the starting position to Start location 1,2.
- Set your AI difficulty (set in the map editor - File -
Editor settings - Test run - Game difficulty) and then press test run or Ctrl + F9
- Don't forget to change the control from "User" to "Computer".

Add. information:

renewing the most popular game StarCraft 2: Wings of Liberty,
making changes and corrections to game process and troubleshooting problems with the game itself.
Designed for the Russian version of the game.
Using this file you can update the game without having to go online,
for example, if you want to install it (the game) on a computer that does not have the ability to connect to the Internet.
List of changes:
update 1.0.3
The Friends tab on the Leagues and Rankings page now works correctly.
Fixed an issue that was preventing some users from accessing single player games without an internet connection.

update 1.0.2
The mechanism for checking victory conditions in campaign missions has been fixed.
Fixed an issue that prevented some users from accessing the mode single player.

update 1.0.1
Optimized the save process in campaign mode.
Fixed an issue with audio playback on some 7.1 systems.