AlphaZero

src: i.ytimg.com

AlphaZero is a computer program developed by the Alphabet-owned AI research company DeepMind, which uses an approach similar to AlphaGo Zero's to master not just Go, but also chess and shogi. On December 5, 2017 the DeepMind team released a preprint introducing AlphaZero, which, within 24 hours, achieved a superhuman level of play in these three games by defeating world-champion programs, Stockfish, elmo, and the 3-day version of AlphaGo Zero, in each case making use of custom tensor processing units (TPUs) that the Google programs were optimized to make use of. AlphaZero was trained solely via "self-play" using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks, all in parallel, with no access to opening books or endgame tables. After just four hours of training, DeepMind estimated AlphaZero was playing at a higher Elo rating than Stockfish; after 9 hours of training, the algorithm decisively defeated Stockfish 8 in a time-controlled 100-game tournament (28 wins, 0 losses, and 72 draws). The trained algorithm played on a single machine with four TPUs.

Video AlphaZero

Relation to AlphaGo Zero

AlphaZero (AZ) is a more generalized variant of the AlphaGo Zero (AGZ) algorithm, and is able to play shogi and chess as well as Go. Differences between AZ and AGZ include:

• AZ has hard-coded rules for setting search hyperparameters.
• The neural network is now updated continually.
• Go (unlike Chess) is symmetric under certain reflections and rotations; AlphaGo Zero was programmed to take advantage of these symmetries. AlphaZero is not.
• Chess can end in a draw unlike Go; therefore AlphaZero can take into account the possibility of a drawn game.

Maps AlphaZero

AlphaZero vs. Stockfish and elmo

Comparing Monte Carlo tree search searches, AlphaZero searches just 80,000 positions per second in chess and 40,000 in shogi, compared to 70 million for Stockfish and 35 million for elmo. AlphaZero compensates for the lower number of evaluations by using its deep neural network to focus much more selectively on the most promising variation.

src: i.ytimg.com

Training

AlphaZero was trained solely via self-play, using 5,000 first-generation TPUs to generate the games and 64 second-generation TPUs to train the neural networks. In parallel, the in-training AlphaZero was periodically matched against its benchmark (Stockfish, elmo, or AlphaGo Zero) in brief one-second-per-move games to determine how well the training was progressing. DeepMind judged that AlphaZero's performance exceeded the benchmark around four hours of training for Stockfish, two hours for elmo, and eight hours for AlphaGo Zero.

src: en.chessbase.com

Results

Outcome

Chess

In AlphaZero's chess tournament against Stockfish 8 (2016 TCEC world champion), each program was given one minute's worth of thinking time per move. Stockfish was allocated 64 threads and a hash size of 1 GB, a setting that Stockfish's Tord Romstad later criticized as suboptimal. AlphaZero was trained on chess for a total of nine hours before the tournament. During the tournament, AlphaZero ran on a single machine with four application-specific TPUs. In 100 games from the normal start position, AlphaZero won 25 games as white, won 3 as black, and drew the remaining 72. In a series of twelve 100-game matches (of unspecified time or resource constraints) against Stockfish starting from the 12 most popular human openings, AlphaZero won 290, drew 886 and lost 24.

Shogi

AlphaZero was trained on shogi for a total of twelve hours before the tournament. In a hundred shogi games against elmo (World Computer Shogi Championship 27 summer 2017 tournament version with YaneuraOu 4.73 search), AlphaZero won ninety times, lost eight times and drew twice. As in the chess games, each program got one minute per move, and elmo was given 64 threads and a hash size of 1 GB.

Go

After 34 hours of self-learning of Go and against AlphaGo Zero, AlphaZero won 60 games and lost 40.

Analysis

DeepMind stated in its preprint that "The game of chess represented the pinnacle of AI research over several decades. State-of-the-art programs are based on powerful engines that search many millions of positions, leveraging handcrafted domain expertise and sophisticated domain adaptations. AlphaZero is a generic reinforcement learning algorithm - originally devised for the game of Go - that achieved superior results within a few hours, searching a thousand times fewer positions, given no domain knowledge except the rules." DeepMind's Demis Hassabis, a chess player himself, called AlphaZero's play style "alien": It sometimes wins by offering counterintuitive sacrifices, like offering up a queen and bishop to exploit a positional advantage. "It's like chess from another dimension."

Given the difficulty in chess of forcing a win against a strong opponent, the 28-72-0 result is a significant margin of victory. However, some grandmasters, such as Hikaru Nakamura and Komodo developer Larry Kaufman, downplayed AlphaZero's victory, arguing that the match would have been closer if the programs had access to an opening database (since Stockfish was optimized for that scenario). Romstad additionally pointed out that Stockfish is not optimized for rigidly fixed-time moves and the version used is a year old.

Similarly, some shogi observers argued that the elmo hash size was too low, that the resignation settings and the "EnteringKingRule" settings (cf. shogi § Entering King) may have been inappropriate, and that elmo is already obsolete compared with newer programs.

src: i.ytimg.com

Reaction

Papers headlined that the chess training took only four hours: "It was managed in little more than the time between breakfast and lunch." Wired hyped AlphaZero as "the first multi-skilled AI board-game champ". AI expert Joanna Bryson noted that Google's "knack for good publicity" was putting it in a strong position against challengers. "It's not only about hiring the best programmers. It's also very political, as it helps makes Google as strong as possible when negotiating with governments and regulators looking at the AI sector."

Human chess grandmasters were very impressed by AlphaZero. Danish grandmaster Peter Heine Nielsen likened AlphaZero's play to that of a superior alien species." Norwegian grandmaster Jon Ludvig Hammer characterized AlphaZero's play as "insane attacking chess" with profound positional understanding. Former champion Garry Kasparov said "It's a remarkable achievement, even if we should have expected it after AlphaGo."

Grandmaster Hikaru Nakamura was less impressed, and stated "I don't necessarily put a lot of credibility in the results simply because my understanding is that AlphaZero is basically using the Google super computer and Stockfish doesn't run on that hardware; Stockfish was basically running on what would be my laptop. If you wanna have a match that's comparable you have to have Stockfish running on a super computer as well."

src: clips-media-assets.twitch.tv

Notes

src: i.ytimg.com

References

src: www.extremetech.com

External links

• Chess.com Youtube playlist for AlphaZero vs. Stockfish

Source of article : Wikipedia