instrumental conditioning
= process whereby organisms learn to make responses unordered to obtain or avoid certain consequences; form of associative learning
Throndike
- law of effect
- Thorndike
- concluded that when an animal’s response was followed by a satisfying consequence, then the probability of that response increased
- law of effect: Stimulus S —> Response R —> Consequence C
Box of Skinner vs Box of Thorndike
- Box of Skinner:
- animal can operate apparatus freely
- Box of Thorndike:
- discrete operat conditioning
classical vs instrumental/operant conditioning
- classical:
- animals receive consequence whether or not they have learned the conditioned response
- instrumental:
- consequence occurs only if the animal performs that response
- more voluntary
experiment of Skinner
- Skinner box
- free-operant paradigm
- reinforcement
Components of Learned Association
- Stimuli
- discriminative stimuli = stimuli that signal whether a particular response will lead to a particular outcome
- habit slip
- Protestant ethic effect = more likely to work for it
Components of Learned Association
- Responses
- oganism learns to give a particular response to obtain (or avoid) a particular consequence
- defined as effect on environment of particular pattern of motor actions
- shaping = technique in which successive approximations to the desired response are reinforced
- chaining = technique in which organisms are gradually trained to execute complicated sequences of discrete responses —> useful for training humans as well
Components of Learned Association
- Consequences
- reinforcer
- primary reinforcers: food, water, sleep, sex
- drive reduction theory = Clark Hull; idea that organisms have innate drives to obtain primary reinforcers and that learning reflects the innate biological need to reduce these drives
- primary reinforcers: food, water, sleep, sex
- punishers
- factors determining how effective punishment will be:
(1) discriminative stimuli for punishment can encourage cheating
(2) concurrent reinforcement can undermine the punishment
(3) initial intensity matters
Building the S-R-C Association
- Timing Affects Learning
- instrumental conditioning is faster if the R-C interval is short
- superstitions ( = responses that individuals make because they believe those responses lead to desired outcomes (or avert undesired outcomes); e.g. rain dances
Building the S-R-C Association
- Instrumental Conditioning Paradigms
- Positive/Negative Reinforcement
- Positive/Negative Punishment
- positive reinforcement = something is added to the environment, and this encourages (reinforces) behavior
- negative reinforcement = something is subtracted from the environment, and this encourages (reinforces) behavior
- positive punishment = behavior is punished by adding something to the environment
- negative punishment = behavior is punished by taking good things away
Building the S-R-C Association
- Schedules of Reinforcement
- continuous reinforcement schedules
- concurrent reinforcement
- partial reinforcement schedules:
- FR: fixed-ratio schedule = some fixed number of responses must be made before a reinforcer is delivered
- FI: fixed-interval schedule = reinfoced that first response after a fixed amount of time
- VR: variable-ratio schedule = e.g. VR5 schedule produces reinfocement after every 5 responses, on average —> responder never knows exactly when a reinfocement is coming —> linked to gambling
Choice Behavior
- matching law of choice behavior
= an organism’s relative rate of responding will (approximately) match the relative rate of reinforcement
Choice Behavior
- behavioral economics
= study of how organisms “spend” their time and effort among possible options
- bliss point = allocation of resources that provides maximal subjective value
Choice Behavior
- Premack principle
= opportunity to perform a highly frequent behavior can reinforce a less-frequent behavior
- response deprivation hypothesis = suggest that the critical variable is not which response is normally more frequent but merely which response has been restricted: by restricting teh ability to execute almost any response, you can make the opportunity to perform that response reinforcing
Brain Substrates
- Basal Ganglia
- stimulus- response associateions may be stored in direct conticocotical connections or indirectly via the basal ganglia
- stimulus inforcation: processed by cortical areas such as V1, S1 and frontal cortex
- voluntary responses: generated by motor cortex (M1)
- instrumental conditioning - basal ganglia helps link associations between sensory and motor cortex, so that stimuli elicit appropriate motor responses
Brain Substrates
- Reinforcement Mechanisms —> VTA
- region in brainstem
- projects dopamine to nucleus accumbens, which in turn projects dopamine to dorsal striatum
- dopaminergic neurons in N.S. project to motor areas in dorsal stiatum that can drive motor responses
Brain Substrates
- Reinforcement Mechanisms —> dopamine
- three theories about extinction mimicry
extinction mimicry = effect in which response of drug-group rats seems to extinguish, even though animals are still receiving food for lever-pressing
- theories:
(1) anhedonia hypothesis = dopamine gives food its “goodness” (hedonic qualities)
(2) incentive salience hypothesis = dopamine helps provide organisms with the motivation to work for reinfocement- increases “wanting” but not “liking”
(3) reward prediction hypothesis = posits that dopamine is involved in predicting future reward
- increases “wanting” but not “liking”
Brain Substrates
- Reinforcement Mechanisms
- opioids and hedonic value
- endogenous opioids = naturally occurring neurotransmitter-like substances (peptides) with many of the same effects as opiate drugs
- mimicked by many highly addictive drugs
- believed to mediate hedonic value
Brain Substrates
- orbitofrontal cortex
-
Clinical Perspectives
- Drug Addiction
- pathological addiction = strong habit (or compulsion) that is maintained despite harmful consequences
- addiction: may involve both seeking “high” (positive reinforcement) and avoiding the averse effects of withdrawal from the drug (negative reinforcement)
- many drugs: opiates —> target opiate receptors in nucleus accumbens, VTA, and other brain areas (e.g. heroin and morphine)
- increasing brain dopamine levels —> e.g. amphetamines and cocaine
Clinical Perspectives
- Behavioral Addiction
= addictions to behaviors rather than drugs, that produce reinforcements or highs, as well as cravings and withdrawal symptoms when the behavior is prevented
- seems that behavioral addictions may reflect dysfunction in the same brain substrates affected by drug addictions
Clinical Perspectives
- Treatments
- cognitive therapy
- e.g. self-help session with support group
- medical treatment
- conditioning-inspired methods:
- extinction = if response R stops producing consequence C, the frequency of R should decline
- distancing = avoiding the stimuli that trigger the unwanted response
- reinfocement of alternate behaviors
- delayed reinforcement
E-Readers
- Internet Addiction
- indicated that some online users were become addicted to the internet in much the same way as others became addicted to drugs or alcohol
- defined as “impulse-control disorder that does not involve an intoxicant”
