chapter 10

Question 1

transfer learning

Answer 1

the ability of a program to transfer what it has learned about one task to help it perform a different, related task.

Unlike humans, none of these programs can “transfer” anything it has learned about one game to help it learn a different game.

Question 2

DeepMind’s most important claim about its results, especially on AlphaGo, is that the work has delivered on that promise (learned from the data, by pure reinforcement learning)

caveats:

Answer 2

A few aspects of human guidance that were critical to its success include;

1. the specific architecture of its convolutional neural network
2. the use of Monte Carlo tree search
3. the setting of the many hyperparameters that both of these entail.

o none of these crucial aspects of AlphaGo were learned from the data, by pure reinforcement learning. Rather, [they were] built in innately … by DeepMind’s programmers.

Question 3

How can we assess how challenging a domain is for AI?

Answer 3

to see how well very simple algorithms perform on it.

Question 4

how can you see that systems aren’t learning like humans

Answer 4

1. they dont understand
   > e.g., inability to generalize
2. vulnerable to adversarial examples

Question 5

From games to the real world: obstacles

Answer 5

1. The need for transfer learning
2. games are clearly delineated
   > have clear rules, straightforward reward functions
3. The more realistic the simulation (to train), the slower it is to run on a computer
4. unpredictability of the real world