Learning

Question 1

Representation

Answer 1

How we represent world so can make decisions about it
Sensory representations
Uncertainty estimation
Bayesian integration

Question 2

valuation

Answer 2

value representations in pfc
Availability/desirability
Assign value, encode diff aspects of value

Question 3

action selection

Answer 3

Striatum - bias process
Action selection in striatum

Question 4

outcome evaluation

Answer 4

Confidence/metacognition

Question 5

learning

Answer 5

Key part, feedback changes preferences
If incorrect = something wrong = estimate how much you should update Your beliefs or represention, or valuation or action selection to get better

Question 6

mice adapt to changes in reward contingencies

Answer 6

Adaptation in learning
Use reward history to assess whether urewarded trials is bc its unrewarded or bc block change
Invert contingencies of task
Lesions - like gambling task, impairment in ability, in leanring to adapt to environment

Question 7

what are the ingredients to build an intelligent agent

Answer 7

Computational approaches = control everything coming in and out fo system and lean rain g process
Need all various competent to learn/be flexible in envir
What part of leanring process is important to learn a certain behaviour

Question 8

computational approach to cognition

Answer 8

agent interact with world
Goal = Maximzie reward
Aspects of environment are represented in diff brain areas
How do observations, rewards, actions change internal model of an agent
What are diff types of leanring in situations where fed observations vs when really interacting with world and receiving reward

Question 9

what is evidence that a behaviour is learned

Answer 9

Action, conditioning, habit
Learning of new Jamir
Improving accuracy = signature fo leanring
Is behaviour innate or learned - motor control

Question 10

nature vs nurture

Answer 10

Giraffe baby can walk immediately
Human baby can’t - learns over time
Complexity of a behaviour is not an indication of leanring vs innate behaviour - lots of structure exists inn the works that can enable complex innate behaviours

Question 11

implicit vs explicit

Answer 11

Learning can be =
Explicit = taught, like read book about physics
Implicit = Jenga, acquire understanding at some point, instinctive - intuitive understanding of physics - gravity

Question 12

motor skill learning and memory

Answer 12

Recall = HM
Acquired specific motor skill but doesn’t remember = improved over days, based on implicit leanring and doesn’t transfer to other hand

Question 13

learning to see and to navigate

Answer 13

Across many domains = many brain processes involved but we cna think about some key concepts that are needed for learning
Is leanring in a given cortex domain general or domain specific (sensory/motor leanring)

Question 14

learning to locate a place

Answer 14

First trial = rat in pool swimming
If swimming = doesn’t see hidden platform so has to explore space and encounter platform
After 10 trials =built map of space and can travel directly to platform = learn where it is
= learn cognitive map fo space

Question 15

learning to discriminate

Answer 15

Have to compare - butterflies
If do task over amd over = will get better

Question 16

what is an essential competent of learning

Answer 16

Memeory
Repetition- test leanring
feed back = internal or external
Adaptation

Question 17

name diff types of tasks

Answer 17

2 main goals = classification or regression
Do them everyday

Question 18

describe classification

Answer 18

Each data point has label
Goal of algorithm = infer label of new data points
Is it a or b
Will it snow tmr

Question 19

describe regression

Answer 19

Each data point has a value
Goal of algorithm = infer value of new data points
More continuous prediction of value
What will temperature be at 12pm tmr

Question 20

types of learning algorithms

Answer 20

Diff types of leanring styles depending on feedback
Distinguished by how data is handled and what signal is for leanring - how is info you have about world processed
How cna they help us understand leanring in the brain - what is type of feedback
What type of learning in diff brain areas at diff stages

Question 21

name the types of learning algorithms

Answer 21

Supervised learning
Unsupervised learning
Reinforcement learning

Question 22

describe learning in artificial networks

Answer 22

Control data, learning rules
Can observe activity of all the units in the network
Can define task
= build everything, compete control of leanring process and capabilities of system
- can help us think about brain

Question 23

describe learning in biological networks

Answer 23

Have some control over data
Don’t control leanring rules
Can observe activity of a very small fraction of units in network - some of neural activity only
Have some control over task
Way harder

Question 24

architecture of convolutional neural network

Answer 24

Simple feature map —> object representations
Hierarchical representation
Train on data and learn

Question 25

What Is supervised learning

Answer 25

Each data point associated with label Goal = learn features fo data that predict the label

Question 26

Supervised learning = training

Answer 26

Each image in training set associated with category = cat or dog Algorithm learns to associate each image in dataset with correct label Have to figure our features

Question 27

Supervised learning = test

Answer 27

If algorithm generalizes well = will also associate new unseen images of dogs and cats to correct label Tell if right or wrong so can update model = build up from sensory pathway

Question 28

credit assignment problem

Answer 28

How do we assign credit to each neuron for its contribution to output of network How do we change connectivity in network to improve the accuracy output = complicated How do we assign credit to some change or part of system to output Train = tell it’s wrong - change neural activity to output the correct identity

Question 29

what is algorithmic rule to update weights in network

Answer 29

Back propagation Used in machine learning Governs how weights are changed given the output of the network and the activity of the units Formation to nudge weights towards giving correct output Compute = relative to how strong error is and how influential given neuron is to network Only possible in artificial systems

Question 30

describe back propagation generally

Answer 30

forwards pass of activity = see something and do this, leads to output Backwards pass of errors = if errors then look back into system and nudge it towards correct answer

Question 31

forward pass

Answer 31

Compute output of net work given the input

Question 32

backwards pass

Answer 32

Update weights in network o reduce the error Structure of network

Question 33

is it easy to measure synaptic changes in biological networks ?

Answer 33

NOO HARD Can only measure pairwaise changes of a few neurons Across a few neurons - have potentiation and depression (depreciation) Weights change in simailr way to artificial network More supervised approach Output - update weights to reach output

Question 34

Measure connectivity between neurons

Answer 34

Measure the connectivity change between 2 neurons = ahve to measure activity of both neurons simultaneously and usually intracellularly (membrane potential rather than spikes)

Question 35

what can cause synaptic changes

Answer 35

Can be due to numerous factors = distribution fo receptors, number of synapses All of these are difficult to measure without access to the membrane potential

Question 36

learning a convolutional neural network

Answer 36

Define architecture of the network - number of layers, feature maps, their sizes etc Use training data and back propagation to learn the weights Test accuracy on a test set of data not used during the training Thinking about tehse supervised approaches = cna help understand how we cold build these representations In humans = not super simailr to how you build your own visual representation of objects

Question 37

Does supervised learning apply to classification or regression

Answer 37

Applies to both Independent of type of task Apply same principle to both cases

Question 38

describe ex = dataset of botanist

Answer 38

Contains random samples of 3 species of flowers Each of species = data set contains 50 obsertaosn for septal length, septal width, petal length and width = how do you classify things when data doesn’t have a label - only have where data lays in space Once you look in more dimensions = see a gradient

Question 39

describe unsupervised learning

Answer 39

Data points unlabelled Goal= to find structure in data Usually done by finding statistical regularity that is indicative of an underlying structure Point are clsoe together inn space of parameters should belong to same category Difficulty= man definitions of clsoe together As the data points are not labelled = this definition of distance is central to the results of the algorithm = clustering challenge = mesure of distance

Question 40

describe und supervised learning = specifically

Answer 40

Not given identity of points Classify and identify the diff types in data Data unlabelled- algorithm learns without supervision Goal = find structure in data What is structure tho - or distance within space of parameters No teaching signal = only internal measure of distance of things

Question 41

what type fo learning happens in brain

Answer 41

Unsupervised=experience world and amke associations Only based on features in data Reinformcent leanring = in between, partial feedback

Question 42

predict grades and train system on labelled things = what type fo learning

Answer 42

Supervised

Question 43

Have everyone in class and all courses they ahve taken= what type fo learning

Answer 43

Unsupervised

Question 44

learning so far

Answer 44

Supervised and unsupervised = presented data and agent had to learn either from labelled or unlabelled exs How does agent learn to act in world Some leanring = extremely fast, one shot, like fear/threat conditioning Info = from actions and learn which actions good

Question 45

what is reinforcement leanring

Answer 45

Learn a policy to take actions that maximize rewards given Rewards sparse, most actions don’t lead directly to a reward How to assign credit to actin in the past that led to a reward

Question 46

define reinforcement leaning

Answer 46

Agent receives occasional rewards and must learn how to act to maximize them

Question 47

maze Ex of reinforcement learning

Answer 47

Only info that right decisions as made = at time of exit Once agent find s exit = can reinforce actions that led to finding the right solution After experiencing maze several times = agent can learn best exit strategy

Question 48

animal behaviour exp that inspired irl

Answer 48

a

Question 49

name the 2 diff kinds of learning

Answer 49

Classical/pavlovian Operant conditoning

Question 50

define classical/pavlovian conditioning

Answer 50

Stimulus given No action necessary Value and associated behavioural response is learned No action of animal

Question 51

define operant condition

Answer 51

Learn to associate response with a cue/state of envir to obtain rewards Have to make voluntary action = lie press lever, need to learn action What is being strengthened = different slightly

Question 52

describe Pavlovian condition = graphs

Answer 52

Acquisition = learn, reaches max Extinction = decouple association = have some sort of forgetting First spontaneous recovery = persistence in memory - ltm happens tho Dynamics in leanring

Question 53

describe Pavlovian condition = specifically

Answer 53

Associate intrinsically rewarding stimulus like food (us) with a simtulus that otherwise has no intrinsic value like sound (cs) Measure behavioural response to cs with leanring across diff conditions Bell has no value but when associated with us (food)= condition stimulus acquires value

Question 54

learning from trial and error = rescorla-Wagner rule

Answer 54

Formalize process Makes some predictions in simple classifications conditioning scenarios= predictions in terms of quantitative changes = how much given reward predicts how much you associate Cs to value - more food =stronger association Why saturates at some point = once predicting value correct, if have correct prediction of value = do not need to increase value

Question 55

limits of r-w rule

Answer 55

Based on trial structure and not easily transferred to sequences of actions and states world is more continuous Ex = thorndikes cat = how do you assign reward credit to any of the specific actions that the cat took in whole sequence Hard to assign credit bc many actions - is it the pressing lever, how do we assign credit to previous actions

Question 56

how do solve a simple navigation task

Answer 56

Reinforcement leanring How can mouse in maze learn which actions are more likely to lead to reward = have to work backwards from reward state Driving to restaurant or putting fork in mouth = both have value bc leads to food, in a state that leads to a state that gives rewards Framework = assign value to state as a function of reward obtained in that state bit also the value of states that can be reached from that state

Question 57

agency in world

Answer 57

Reinforcement learning How do you link everything you did before to the feedback and how do you maximize reward

Question 58

Perception

Answer 58

visual representations in ventral stream

Question 59

value functions

Answer 59

value representations in pfc

Question 60

reactive policies

Answer 60

biasing action selection in striatum From perception to decision in dorsal stream - Mt/lip

Question 61

transition model

Answer 61

bayes rule to update world model