Google’s Go-playing AI system beat Korean grandmaster 4 out of 5 games

17 March 2016

share this:

Machines have been beating humans for years in games and contests: chess, Scrabble, checkers, Jeopardy! and more recently, the Chinese board game Go. Google’s AI machine won 4 out of 5 games in a $1.3 million contest against South Korean Lee Sedol, rated as the world’s best Go player. Sedol comments to ABC News at a post-game press conference:

I didn’t expect to lose. I didn’t think AlphaGo would play the game in such a perfect manner. Personally, I am regretful about the result, but would like to express my gratitude to everyone who supported and encouraged me throughout the match.

The 2,500-year-old board game is purported to be more complex than chess. It is played with black and white stones and the aim is to surround more territory than the opponent. Go appears to be a simple game and begins with an empty board. Two players (one using black stones, the other white), alternate placing stones in squares, trying to grab territory without getting their pieces captured.

According to Alan Levinovitz, writing in Wired a few years ago, “there are 400 possible board positions after the first round of moves in Chess and 129,960 in Go. There are 35 possible moves on any turn in a Chess game, and 250 for Go.” DeepMind’s David Silver and Demis Hassabis noted that the number of possible board configurations in Go is larger than the number of atoms in the universe.

Drake Baer, in a Tech Insider piece, describes DeepMind’s history and role in the development of AlphaGo:

DeepMind didn’t “program” AlphaGo with evaluations of  “good” and “bad” moves. Instead, AlphaGo’s algorithms studied a database of online Go matches, giving it the equivalent experience of doing nothing but playing Go for 80 years straight.

“This deep neural net is able to train and train and run forever on these thousands or millions of moves, to extract these patterns that leads to selection of good actions,” says Carnegie Mellon computer scientist Manuela Veloso, who studies agency in artificial intelligence systems.

Google acquired DeepMind in 2014. Founded in 2010 by chess prodigy-turned-artificial intelligence researcher Demis Hassabis, the company’s mission is to “solve intelligence,” and it claims that “the algorithms we build are capable of learning for themselves directly from raw experience or data.”

In February 2015, DeepMind revealed in Nature that the program learned to play vintage arcade games like Pong or Space Invaders as well as human players. Now it’s about to master a game that once seemed unmasterable for artificial intelligence [Go].

A Google DeepMind paper describes the technical details. You can also watch the Nature video below with DeepMind’s Demis Hassabis, who then describes the process required by the AI.

What does it all mean?

Go’s appeal lies in depth through simplicity, which is why its difficult for computers to master up — until now. There’s limited data available looking at the board. Choosing a good move demands a great deal of intuition. Until AlphaGo, no one had built an effective evaluation function. But AlphaGo’s use of deep learning and neural networks to teach itself enables it to process millions of Go positions and moves from human-played games, then deciding upon moves predicted on that experience.

Sam Byford writes in The Verge:

The twist is that DeepMind continually reinforces and improves the system’s ability by making it play millions of games against tweaked versions of itself. This trains a “policy” network to help AlphaGo predict the next moves, which in turn trains a “value” network to ascertain and evaluate those positions. AlphaGo looks ahead at possible moves and permutations, going through various eventualities before selecting the one it deems most likely to succeed. The combined neural nets save AlphaGo from doing excess work: the policy network helps reduce the breadth of moves to search, while the value network saves it from having to internally play out the entirety of each match to come to a conclusion.

Thus, AlphaGo gets better by playing itself. DeepMind believes that the principles it uses in AlphaGo have broader application and implications. Hassabis makes a distinction between “narrow” AIs like Deep Blue and artificial “general” intelligence (AGI), the latter being more flexible and adaptive. He thinks AlphaGo’s machine learning techniques will be useful in robotics, smartphone assistant systems, and healthcare.

Intuition has been the sole domain of humans. But now AlphaGo, Google, and DeepMind have shown they can build systems with an effective evaluation function to make choices based on a growing universe of experiential data. Reading medical scans and upgrading Siri (and other online resources and social assistants) are just the tip of the iceberg. This technology has implications for self-driving vehicles of all types (air, land and sea), medical diagnostics, and predictive stock picking to name just a few.

If you liked this article, you may also enjoy:

tags: , , , ,

Frank Tobe is the owner and publisher of The Robot Report, and is also a panel member for Robohub's Robotics by Invitation series.
Frank Tobe is the owner and publisher of The Robot Report, and is also a panel member for Robohub's Robotics by Invitation series.

Related posts :

Robot Talk Episode 90 – Robotically Augmented People

In this special live recording at the Victoria and Albert Museum, Claire chatted to Milia Helena Hasbani, Benjamin Metcalfe, and Dani Clode about robotic prosthetics and human augmentation.
21 June 2024, by

Robot Talk Episode 89 – Simone Schuerle

In the latest episode of the Robot Talk podcast, Claire chatted to Simone Schuerle from ETH Zürich all about microrobots, medicine and science.
14 June 2024, by

Robot Talk Episode 88 – Lord Ara Darzi

In the latest episode of the Robot Talk podcast, Claire chatted to Lord Ara Darzi from Imperial College London all about robotic surgery - past, present and future.
07 June 2024, by

Robot Talk Episode 87 – Isabelle Ormerod

In the latest episode of the Robot Talk podcast, Claire chatted to Isabelle Ormerod from the University of Bristol all about human-centred design and women in robotics.
31 May 2024, by

Robot Talk Episode 86 – Mario Di Castro

In the latest episode of the Robot Talk podcast, Claire chatted to Mario Di Castro from CERN all about robotic inspection and maintenance in hazardous environments.
24 May 2024, by

Congratulations to the #ICRA2024 best paper winners

The winners and finalists in the different categories have been announced.
20 May 2024, by

Robohub is supported by:

Would you like to learn how to tell impactful stories about your robot or AI system?

training the next generation of science communicators in robotics & AI

©2024 - Association for the Understanding of Artificial Intelligence


©2021 - ROBOTS Association