Value Based Decision Making I

Question 1

Which brain areas are involved in computing value of options and states of world
How can we study value across domains

Answer 1

Assign values to different states of world with goal of maximizing value of things we do
Pfc = where these value computations happen
Striatum = arbitration between value and basing behaviour towards choosing a type of value over another

Question 2

perceptual discrimination task

Answer 2

Changing difficulty of stimulus to understand transformation from sensory perception to decision formation and evaluation
#of dots, coherence, gratings etc
Then identify and characterize —> brain areas, neurons over stages (cognitive sensory)

Question 3

Free choice task

Answer 3

Decision rely on subjective value between options
Internally generated
How do we construct such subjective value representations and how do we arbitrate between them
Assign value -can we put on common axis
Which brain circuits underly tehse computations

Question 4

describe Iowa gambling task

Answer 4

4 decks
A-b = 100 reward p=1, 250 loss p=0.5 (more risky, lose money on avg in long run)
C-d = 50 reward p=1, 50 loss p=0.5 (safer option, win money in long run )

Question 5

what does Iowa gambling task measure

Answer 5

ability to integrate rewards and losses in a probabalisitic setting

Question 6

measuring decision differences across subjects and patients = compare Iowa and Wisconsin

Answer 6

Lesion vmpfc
No impairment in. Control tasks and in Wisconsin task (match to rule- changes and have to figure out rule change, error means rule switched)
In Iowa task = challenge is about estimating the average long term reward in a probabilistic setting rather than the rule that provides a guaranteed reward
Obvious impairments in daily life

Question 7

Describe what happens when vmpfc lesion in Iowa task = control

Answer 7

Start by choosing risk decks before reverting to choosing safe decks bc have positive avg reward in long run

Question 8

Describe what happens when vmpfc lesion in Iowa task = lesion group

Answer 8

Did not switch behaviour
Choose high risk more often and cannot switch
Domain of info diff = integrate probabilities over time to be able to determine that one set of decks is better than other

Question 9

Describe what happens when vmpfc lesion in Iowa task= conclusions

Answer 9

Simailr to patients in Wisconsin sortinggame where patients had an inability to change their behaviour with new info
But here domain diff = estimating probabilistic rewards rather than rules

Question 10

what do we have to consider = patient cohort to investigate localization and laterality

Answer 10

Cohort of patients wirh lesions across pfc areas
Group patients by overlap of lesionned areas with regions of interest - some overlap, some large, soma small
Also study effect of laterality = bi vs uni lateral
Important to consider heterogeneity of the origin of the lesion across the cohort= this, blunt force trauma, stroke

Question 11

describe results patient cohort to investigate localization and laterality= all patients

Answer 11

Typical subjects start with risky decks and switch to safer ones
Frontal patients do switch but then return to risky option

Question 12

describe results patient cohort to investigate localization and laterality= left vs right control lesions

Answer 12

Score deficits appear regardless of lesionned hemisphere
Same results for right vs left frontal hemispheres = not lateralzied

Question 13

describe results patient cohort to investigate localization and laterality= localizaiton of lesion

Answer 13

Dorsolateral, venteromedial, extended (both areas)
No significant effect of prefrontal area lesionned

Question 14

describe results patient cohort to investigate localization and laterality= conslusions

Answer 14

Vmpfc involved but not unique contribution = if lesion whole Pfc will still have same impairments in value based decision making

Question 15

broad activation across pfc in task = compare what

Answer 15

Compare relative activation between Iowa gambling task and a control task where the outcome of each deck is explicitly given
= task structure the same but gambling and payoff estimation under uncertainty component is removed
= no building probabailsitic represnettaion = remove subjective valuation

Question 16

broad activation across pfc in task = results

Answer 16

Monitoring approach
Broad activation of Pfc and other areas including ventral striatum (downstream action selection)

Question 17

What guides decisions

Answer 17

perception and valuation
- identify something more specific to vmpfc
What types of value representations are being done where

Question 18

stages of value based decision making = gen

Answer 18

Represent possible states of world
Assign value to each state - how to compare across states
Select action
Evaluate outcomes
Update valuation of options

Question 19

stages of value based decision making = stage 1

Answer 19

Representation of states of world
Set of feasible actions, internal and external states

Question 20

stages of value based decision making = stage 2

Answer 20

Valuation
What is value of each action given internal and external states - like if thirsty, water bottle has more value

Question 21

stages of value based decision making = stage 3

Answer 21

Action selection
Choose actions based on valuations
Picks best option to pick another option to try out diff things

Question 22

stages of value based decision making = stage 4

Answer 22

Outcome evolution = how desirable are the outcomes and states that followed the action - confidence and meta cognition, is outcome good
Then —> learning = update representation, valuation and action selection processes = adapt

Question 23

pavlovian valuation system

Answer 23

Associated with something good, direct reward linked to action we did
Appetitive= eat food, food reward
Avoidance= cross street when see dangerous person, avoid potential harm

Question 24

Habitual valuation system

Answer 24

Do bc used to doing it, learned behaviour from many actions, habit, not direct rewards
Appetitive = coffee in morning, stimulant reward
Avoidance = drive usual route to work, avoid traffic

Question 25

Goal directed valuation system

Answer 25

Not immediate, long term goal working towards Appetitive = movie, get entertained and also convo starter/knowledge Avoidance = go for run, avoid obesity

Question 26

describe different types of valuation

Answer 26

Diff value systems are often in conflict with each other in most real life situations = compete within valuation processes and also across Isn’t just the way thes values have been learned that is diff but also their nature How Do we integrate these conflicting values to make deliciousness Still have to compare and constrast Put on axis and compare = common currency like exchange rate

Question 27

pavlovian vs Pavlovian

Answer 27

Hungry animal = food and shock at same time Appetitive = high value for food, low value for escape behaviours Avoidance = high value for escape and low value for food

Question 28

pavlovian vs habitual

Answer 28

animal rewarded for running away from food Appetitive Pavlovian = high value for running towards food Avoidance habitual system = high value for running away from food = will learn

Question 29

pavlovian vs goal directed

Answer 29

Individual considering taking extra bit after full Appetitive Pavlovian = high value for food Avoidance health goal directed = low value for food

Question 30

habitual vs habitual

Answer 30

Animal trained to run towards lever in response to a sound and away from lever in response to a light being presented with both stimuli = learned 2 conditions by habit = not intrinsic Appetitive habitual = high value for lever approach Avoidance habitual = high value for lever avoidance

Question 31

habitual vs goal directed

Answer 31

Alcoholic considering having a drink at bar Appetitive habitual = high value for drink Avoidance goal directed = low value for drink

Question 32

goal directed vs goal directed

Answer 32

Dieter considering having ice cream Appetitive goal directed = high value for ice cream Avoidance = low value for ice cream

Question 33

identifying neural processes underlying diff stages of decision process

Answer 33

Stage 1 = valuation of options, predicted valuation signals, compare, what you rate expecting to get for each option Stage 2 = choice, action valuation signals, might be diff depending on context, independent of how you choose value Stage 3 = outcome, experienced valuation signals, use to create feedback and refine and improve subjective learning Have to do across modalities Stage 1 and 3 = vmpfc

Question 34

comapring social and monetary rewards = set up

Answer 34

Rewards can have diff natures - measure choices to slot machines with diff probabilities of outcomes = monetary -1,0,+1 or social outcomes happy, neutral, angry Measure behavioural preference to diff options Structure of task the same, change what making the valuation about Use task approach to compare monetary and social rewards Can construct measure of decision difficulty by comparing value of both options = value (option 1)- value (option 2)

Question 35

comapring social and monetary rewards = graphs

Answer 35

Quickly figured out that green best option then orange then purple For both the social and monetary rewards Subject prefer high value items irrespective of the modality Both monetary and social outcomes can ve placed on a decision axis - simailr to psychometric, less difference = more of an ambiguous trick, at ends = much more likely to pick one with 80% reward Based on value axis = likelihood of getting a good outcome on the choice measure predictability of subjective preference of person, trail by trial

Question 36

Common representations across reward modalities to subjective value and reward values = describe model

Answer 36

How correlated is brain activity to subjective value - measure, in areas more activated on certain tasks, if there’s higher subjective value more active? Model allows us to identify brain activity correlated to the subjective value - in which brain areas to changes in activity across trails correlate with changes in subjective value

Question 37

Common representations across reward modalities to subjective value and reward values = describe results kinda

Answer 37

In both tasks - value representation there is really correlated with subjective value, so activity of brain correlated with subjective value for both modalities Overlap in representations across modalities in the vmpfc = common currency for values? Common currency computing - project onto 1 axis of decision, what is value of object

Question 38

measuring preferences across modalities

Answer 38

Mixed type of decision Ppl have diff risk preferences —> measure preference to probability and to size of something Measuring choices to diff amounts and risk of diff options across modalities within same choices Diff types rewards offered wirh diff probabilities and subjects make choices Can use choices to quantify how diff reward attributes (modality, size, probability) affect their values

Question 39

quantitative measurement of preferences and risk across modalities = describe model

Answer 39

Fit a behavioural model to subjects choices Measure behaviour and infer parameters Model allows us to map the objective value to the subjective value for each obejct and each decision - use model and see if brain areas represent functions

Question 40

quantitative measurement of preferences and risk across modalities = 2 main parameters

Answer 40

Quantity = amount fo reward, how much it affects subjective value Risk = probability of receiving the reward, how much it affects subjective value

Question 41

quantitative measurement of preferences and risk across modalities = mapping risk

Answer 41

map probability of obtaining chosen option = risk Affects the probability of choosing the option More likely you are to obtain it the more likely you are to choose it Measuring risk carefully across modalities, allows us to compare valuation across modalities Quantify risk taking = measure empirically risk ppl wiling to take for these diff quantities Changing size and their probabilities If 3% chance = need to have a lot of money to take option, but if money guaranteed = only need bit of money to pick option

Question 41

quantitative measurement of preferences and risk across modalities = utility curve

Answer 41

Measures how quantity of reward is related to subjective value for this reward = how subjective utility changes for diff amounts of stuff Relationship typically non linear, sublinear = twice as much of something is better than one but not exactly twice as good Preference to size

Question 42

quantitative measurement of preferences and risk across modalities = overall

Answer 42

Have 2 quantities = quantity and risk = now have subjective value and can compute And once have subjective value = look at what we see in brain

Question 43

common representations across reward modalities = what does model allow us to obtain

Answer 43

Obtain esv=estimated subjective value for reach option and find neural activity correlated to it Analysis can be performed independently as a function of reward modality and identifies brain areas with overlapping representations - more subjective value = more active

Question 44

common representations across reward modalities = results conclusions

Answer 44

Vmpfc appears to encode a representation of value across modalities Location of a valuation computation in a common currency = allowing to compare values across modalities Overlap of value representations = represents value indeed entry of how it was built - compare with other values Abstract vision of value then can compare = common currency Related to action selection