Clan ThE)

Building a StarCraft bot that plays like a Pro

PhD students at UC Santa Cruz are working on building an artificial intelligence designed to take on the very best StarCraft players in the world. Here they explain how they're doing it, and the challenges that lie ahead for them.

Can an artificial intelligence play StarCraft like a professional player? That's what UC Santa Cruz students Ben Weber and Peter Mawhorter set out to test when they began their latest project at the university's Expressive Intelligence Studio. Working with their associate professor, the noted interactive drama pioneer Michael Mateas, they are participating in the studio’s broader goal of exploring the intersection of artificial intelligence, art, and design. According to Mateas, "our goal is to create compelling new forms of interactive art and entertainment that provide more deeply autonomous, generative and dynamic responses to interaction. A major thrust of this work is advanced A.I. for video games, including autonomous characters and interactive storytelling. By viewing A.I. as an expressive medium, our work raises and answers novel A.I. research questions while pushing the boundaries of the conceivable and possible in interactive experiences. Current projects in the group include automated support for game generation, automatic generation of autonomous character conversations, story management, and authoring tools for interactive storytelling."

"Real-time strategy games provide an excellent environment for A.I. research and creating bots that are capable of defeating skilled players in this domain is still an open problem," explains Weber. "EISBot is the Expressive Intelligence Studio's StarCraft bot and is part of our dissertation research. It is coded in a reactive planning language and is composed of managers that handle different aspects of gameplay. EISBot selects build orders from a set of replays using case-based reasoning. Our goal is to build a bot that learns how to play StarCraft competitively based on analysis of expert StarCraft replays."

The bot, which currently plays Protoss exclusively, and is already enjoying a sustained 20 percent win-rate against human players, is a development of a previous UC Santa Cruz project that used reactive planning to play the open source WarCraft II clone Wargus. The code also makes use of the custom A.I. language that was developed for authoring characters in Mateas' interactive drama Facade, which was an artificial intelligence-based story that blended video game elements with natural language processing. The game won the Grand Jury Prize at the 2006 Slamdance Independent Games Festival.

"We picked StarCraft specifically because of the active community," Weber tells us, when asked why he and Mawhorter selected the game for their research, versus other alternatives. "In South Korea, hundreds of professional gamers actively participate in tournaments such as the MBC and OGN star leagues. This community generates a large amount of replays that are available for building bots. It also generates a large number of interesting StarCraft matches to watch. StarCraft is played all over the world and there are several active community websites. It's easy to find players interested in playing the EISBot and our bot has already played against players in over 30 countries," he explains enthusiastically.

"Another reason we selected StarCraft was because of the complexity of the game,” Mawhorter adds. "StarCraft has three distinct races and is a well-balanced game; in any particular match up in StarCraft, there are a large number of strategies that are valid. Despite being over 10 years old, StarCraft does not have a dominant strategy. StarCraft has an active meta-game in which popular strategies are constantly evolving. This factor adds to the challenge of building strong computer opponents."

Ultimately, the team's goal is to achieve "human level behavior," says Weber. They want to be able to build a bot that can not only consistently win individual games, but one that is also capable of tracking the evolving meta-game. "New strategies are constantly being uncovered by players and playing competitively requires learning how to counter newly discovered strategies," he explains. "Our approach to this problem is to build a bot that learns new strategies from replays. Therefore, by giving it more replays, you can expand the strategic possibilities that it considers."

But if the bot simply learns from other players' behavior, isn't it going to be a predictable opponent to really experienced players? What about "flair" or unpredictable behavior that fools less experiences players. Can a bot trick human players? “The bot learns strategies from a collection of replays," says Weber. "If the bot is given a large number of replays, then it will execute several different strategies. Since the bot chooses strategies from the set, it will not intentionally execute novel builds, unless there is an example in the replays. One of our eventual goals is to achieve stylized gameplay. If we give the bot replays from a specific professional gamer, then the bot will choose strategies that the pro-gamer commonly executes.

"We have not yet achieved stylized gameplay or 'flair' at the tactical level though," Weber confesses. "This is because the tactical knowledge in our bot is hand-authored, and is not learned from replays. For example, our bot does not currently execute distraction attacks or multi-pronged attacks. EISBot currently lacks knowledge of Sun Tzu's Art of War."

So, returning to our original question; can a bot really play like a pro? "Building A.I. for StarCraft has been a humbling experience," says Weber. "It's difficult to build a competitive bot. A lot goes on in a competitive player's mind during a game and it is difficult to encode all of that knowledge. We expect that building strong computer opponents for StarCraft will require utilizing an ensemble of innovative A.I. techniques. While our research goal is to build A.I. that learns automatically, there is a limit to how much can be learned. EISBot currently learns build orders, but does not learn micro-management strategies such as dancing (the act of retreating while units are in cool down, and forcing opponent troops to stop attacking and chase them.) There are specific situations where we want the bot to perform a specific action and have to hand-author part of the bot's behavior."

The bottom line? "It's going to take a lot more work to reach this goal. One thing to consider is how invested professional gamers are in StarCraft. It is common for pro gamers to practice over 60 hours a week during peak training. Building an A.I. capable of defeating players of this caliber is a challenging task."

Ultimately, the team's goal is to achieve "human level behavior," says Weber. They want to be able to build a bot that can not only consistently win individual games, but one that is also capable of tracking the evolving meta-game. "New strategies are constantly being uncovered by players and playing competitively requires learning how to counter newly discovered strategies," he explains. "Our approach to this problem is to build a bot that learns new strategies from replays. Therefore, by giving it more replays, you can expand the strategic possibilities that it considers."