Google’s Go-playing AI system beat Korean grandmaster 4 out of 5 games

17 March 2016

share this:

Machines have been beating humans for years in games and contests: chess, Scrabble, checkers, Jeopardy! and more recently, the Chinese board game Go. Google’s AI machine won 4 out of 5 games in a $1.3 million contest against South Korean Lee Sedol, rated as the world’s best Go player. Sedol comments to ABC News at a post-game press conference:

I didn’t expect to lose. I didn’t think AlphaGo would play the game in such a perfect manner. Personally, I am regretful about the result, but would like to express my gratitude to everyone who supported and encouraged me throughout the match.

The 2,500-year-old board game is purported to be more complex than chess. It is played with black and white stones and the aim is to surround more territory than the opponent. Go appears to be a simple game and begins with an empty board. Two players (one using black stones, the other white), alternate placing stones in squares, trying to grab territory without getting their pieces captured.

According to Alan Levinovitz, writing in Wired a few years ago, “there are 400 possible board positions after the first round of moves in Chess and 129,960 in Go. There are 35 possible moves on any turn in a Chess game, and 250 for Go.” DeepMind’s David Silver and Demis Hassabis noted that the number of possible board configurations in Go is larger than the number of atoms in the universe.

Drake Baer, in a Tech Insider piece, describes DeepMind’s history and role in the development of AlphaGo:

DeepMind didn’t “program” AlphaGo with evaluations of  “good” and “bad” moves. Instead, AlphaGo’s algorithms studied a database of online Go matches, giving it the equivalent experience of doing nothing but playing Go for 80 years straight.

“This deep neural net is able to train and train and run forever on these thousands or millions of moves, to extract these patterns that leads to selection of good actions,” says Carnegie Mellon computer scientist Manuela Veloso, who studies agency in artificial intelligence systems.

Google acquired DeepMind in 2014. Founded in 2010 by chess prodigy-turned-artificial intelligence researcher Demis Hassabis, the company’s mission is to “solve intelligence,” and it claims that “the algorithms we build are capable of learning for themselves directly from raw experience or data.”

In February 2015, DeepMind revealed in Nature that the program learned to play vintage arcade games like Pong or Space Invaders as well as human players. Now it’s about to master a game that once seemed unmasterable for artificial intelligence [Go].

A Google DeepMind paper describes the technical details. You can also watch the Nature video below with DeepMind’s Demis Hassabis, who then describes the process required by the AI.

What does it all mean?

Go’s appeal lies in depth through simplicity, which is why its difficult for computers to master up — until now. There’s limited data available looking at the board. Choosing a good move demands a great deal of intuition. Until AlphaGo, no one had built an effective evaluation function. But AlphaGo’s use of deep learning and neural networks to teach itself enables it to process millions of Go positions and moves from human-played games, then deciding upon moves predicted on that experience.

Sam Byford writes in The Verge:

The twist is that DeepMind continually reinforces and improves the system’s ability by making it play millions of games against tweaked versions of itself. This trains a “policy” network to help AlphaGo predict the next moves, which in turn trains a “value” network to ascertain and evaluate those positions. AlphaGo looks ahead at possible moves and permutations, going through various eventualities before selecting the one it deems most likely to succeed. The combined neural nets save AlphaGo from doing excess work: the policy network helps reduce the breadth of moves to search, while the value network saves it from having to internally play out the entirety of each match to come to a conclusion.

Thus, AlphaGo gets better by playing itself. DeepMind believes that the principles it uses in AlphaGo have broader application and implications. Hassabis makes a distinction between “narrow” AIs like Deep Blue and artificial “general” intelligence (AGI), the latter being more flexible and adaptive. He thinks AlphaGo’s machine learning techniques will be useful in robotics, smartphone assistant systems, and healthcare.

Intuition has been the sole domain of humans. But now AlphaGo, Google, and DeepMind have shown they can build systems with an effective evaluation function to make choices based on a growing universe of experiential data. Reading medical scans and upgrading Siri (and other online resources and social assistants) are just the tip of the iceberg. This technology has implications for self-driving vehicles of all types (air, land and sea), medical diagnostics, and predictive stock picking to name just a few.

If you liked this article, you may also enjoy:

tags: , , , ,

Frank Tobe is the owner and publisher of The Robot Report, and is also a panel member for Robohub's Robotics by Invitation series.
Frank Tobe is the owner and publisher of The Robot Report, and is also a panel member for Robohub's Robotics by Invitation series.

Related posts :




NVIDIA and ROS Teaming Up To Accelerate Robotics Development, with Amit Goel

Amit Goel, Director of Product Management for Autonomous Machines at NVIDIA, discusses the new collaboration between Open Robotics and NVIDIA. The collaboration will dramatically improve the way ROS and NVIDIA's line of products such as Isaac SIM and the Jetson line of embedded boards operate together.
23 October 2021, by

One giant leap for the mini cheetah

A new control system, demonstrated using MIT’s robotic mini cheetah, enables four-legged robots to jump across uneven terrain in real-time.
23 October 2021, by

Robotics Today latest talks – Raia Hadsell (DeepMind), Koushil Sreenath (UC Berkeley) and Antonio Bicchi (Istituto Italiano di Tecnologia)

Robotics Today held three more online talks since we published the one from Amanda Prorok (Learning to Communicate in Multi-Agent Systems). In this post we bring you the last talks that Robotics Today...
21 October 2021, by and

Sense Think Act Pocast: Erik Schluntz

In this episode, Audrow Nash interviews Erik Schluntz, co-founder and CTO of Cobalt Robotics, which makes a security guard robot. Erik speaks about how their robot handles elevators, how they have hum...
19 October 2021, by and

A robot that finds lost items

Researchers at MIT have created RFusion, a robotic arm with a camera and radio frequency (RF) antenna attached to its gripper, that fuses signals from the antenna with visual input from the camera to locate and retrieve an item, even if the item is buried under a pile and completely out of view.
18 October 2021, by

Robohub gets a fresh look

If you visited Robohub this week, you may have spotted a big change: how this blog looks now! On Tuesday (coinciding with Ada Lovelace Day and our ‘50 women in robotics that you need to know about‘ by chance), Robohub got a massive modernisation on its look by our technical director Ioannis K. Erripis and his team.
17 October 2021, by

©2021 - ROBOTS Association


©2021 - ROBOTS Association