1
Q
Operant Behavior
A
• Pavlovian responses are passive; they happen to you (reflexes, autonomic reactions).
• Operant behavior is active; it consists of the things you do.
Pavlovian responses are elicited(by stimuli) Operant behavior is emitted(by an organism)
• B. F. Skinner coined the term “operant behavior” to characterized the behavior studied by his influential predecessor, Edward Thorndike.
• Many of Skinner’s early ideas were derived from Thorndike’s work, but Skinner developed his own terminology and philosophical perspective (radical behaviorism).
2
Q
Thorndike: The Law of Effect
A
- Thorndike’s research emphasized the effects on learning of the consequences that follow behavior.
- Law of Effect: Behavior followed by favorable outcomes will increase in frequency; behavior followed by undesirable outcomes will decrease.
- Therefore, he was the first scientist to systematically study operant behavior, although he called the changes that occurred trial-and error learning.
3
Q
Skinner’s operant learning is not S-R (reflex) learning
A
• Early behaviorists like John Watson used the reflex terminology of stimulus-response (S-R) psychology to account for all behaviors.
• From this perspective, stimuli force responses much like meat in a dog’s mouth forces salivation.
• Using the reflexive, S-R approach leads to awkward, mechanistic explanations of voluntary, goal-oriented behaviors.
• Example on board of Clark Hull’s S-R chaining
– Every point in a maze is a stimulus that drives an organism reflexively to the reinforcement at the end of a maze.
4
Q
Molecular vs Molar Behaviorism
A
Watson and Hull’s early Molecular approach:
All behavior, even operant behavior,is made up of S-R chains.Complex behavior = more complex chains.
Tolman & Skinner’s molar approach:
Behavior is made up of learned,whole, integrated acts that can be extended over time in complex ways. No need to describe or account for every twitch: just describe the final behavioral outcome—the rat runs to the end of the maze more rapidly than before.
5
Q
S-R vs R-S learning
A
- Skinner agreed that classical (Pavlovian) conditioning could be characterized as S-R learning.
- But not all learning was S-R learning.
- Operant conditioning might be better described as R-S learning
- Complex behaviors are emitted first (R)
- This response leads to an environmental outcome (S), which then determines whether R is likely to be repeated.
6
Q
The Selection of Operant Behavior
A
- Skinner preferred to describe operant conditioning in terms borrowed from Darwin’s natural selection.
- The principle of selection of behavior states that an individual emits behavior that produces effects, consequences, or outcomes.
- Based on these consequences,some behaviors increase in an organism’s behavioral repertoire while other behaviors decline or become extinct.
- (Analogous to physical traits increasing or decreasing in a population due to natural selection)
- Try something (behave); keep what works (i.e. what brings reinforcement)
7
Q
Methodological Advances
A
• Discrete Trial Procedures: Behavior of an organism ends a trial (e.g., escape from box, or finding the end of a maze ends a trial; start again)
– Requires researchers to be present during task.
• Skinner introduced “Free Operant Procedures” : Ongoing operant behavior can be “freely” repeated any number of times; rates of responding automatically recorded by a cumulative recorder.
– Researchers can do other things while behavior is recorded automatically.
8
Q
Operant Conditioning
3
A
• Any procedure through which an operant behavior becomes stronger or weaker depending on the consequences of that behavior.
–Strength of behavior refers to its frequency, rate, probability of occurrence, intensity, or persistence.
9
Q
Contingency
A
• In operant conditioning, we say that the strength of an operant behavior is contingent upon its consequences.
–Recall that contingency implies an if-then relationship.
• In operant conditioning, if I do this, then a certain consequence follows.
• If behavior, then consequence.
10
Q
Operant Conditioning Procedures
A
- Reinforcement: providing consequences that increase or maintain the strength of an operant behavior.
- Punishment: providing consequences that reduce the strength of an operant behavior.
11
Q
Environmental Consequences
A
• Consequences of a behavior can be described in terms of whether an environmental stimulus is presented or removed as a result of a behavior.
–Sometimes our actions result in something becoming present or available to us.
–Think (+) (addition)
–Sometimes our action result in something being removed or taken from us.
–Think (−) (subtraction)
12
Q
Two types of Reinforcement
A
Positive Reinforcement: A response is strengthened by adding a stimulus or increasing its intensity
(e.g., + food, praise)
The stimulus is called a positive reinforcer.
Negative Reinforcement: A response is strengthened by subtracting a stimulus or decreasing its intensity
(e.g.,—shock, nagging)
The stimulus is called a negative reinforcer.
13
Q
Positive reinforcers as rewards
A
- Positive reinforcers are often things we seek out, things that bring us pleasure or satisfaction, things that might be called rewards for our behavior.
- Sometimes positive reinforcement is called reward learning.
- Skinner objected: “People are rewarded, but behavior is reinforced.”
- What is positively reinforcing must be determined empirically: does its presentation increase the strength of behavior?
14
Q
Negative reinforcement and escape
A
- Negative reinforcers are often things we find aversive, things we would normally avoid.
- Sometimes negative reinforcement is called escape-avoidance learning.
- We will do something more often (strengthen a behavior) to avoid or escape something we don’t like.
- As with positive reinforcement, the stimuli that act as negative reinforcers must be determined empirically for a particular species or organism.
15
Q
Punishment
A
Any stimulus or outcome that decreases the strength of an operant behavior.
–Positive Punishment: Decreases the likelihood of a behavior by presenting (adding) a stimulus (usually aversive).
–Negative Punishment: Decreases the likelihood of a behavior by removing (subtracting) a stimulus (usually a desirable one).
16
Q
Kinds of Reinforcers: Primary & Secondary
A
• Primary reinforcer: any reinforcer that is not dependent on another reinforcer for its reinforcing properties.
–Often biologically based; innately or naturally reinforcing for members of a species.
–Food, water, sexual stimulation, or relief from pain, cold, or other aversive conditions.
• Secondary Reinforcer: Any reinforcer that has acquired its reinforcing properties through its association with other reinforcers.
–Also called “conditioned reinforcers”
• e.g., pair a buzzer with food (classical conditioning)
• Then the buzzer can be used to reinforce operant behavior (such as pressing a bar).
17
Q
Generalized Reinforcers
A
- Any secondary reinforcer that has been paired with a variety of other reinforcers.
- Money: Paired with almost every imaginable reinforcer in consumer societies.
- In prisons, cigarettes are generalized reinforcers.
- In some institutions, tokens are generalized reinforcers (token economies).
18
Q
Kinds of Reinforcers
A
–Contrived (“extrinsic”) reinforcers: Any reinforcing event that has explicitly been designed to modify behavior.
• Almost all reinforcements in research and applied behavior analysis.
–Natural (“intrinscic”) reinforcers: Any reinforcing event that follows automatically, i.e., naturally from a behavior.
• Pleasure from succeeding on a task.
• Scratching an itch.
• Reading for intrinsic pleasure (natural) vs. reading to earn a star in class (contrived).
19
Q
Social Reinforcers
A
• Social reinforcers are consequences that involve personal communication from others
– Praise, touch, compliments
– Many mammals respond to these social reinforcers( Limbic System): “good dog” in a happy, warm tone. ‘
• Social reinforcers are easy to deliver, can be applied immediately, and are not very disruptive
20
Q
Shaping & Chaining
A
• By definition, reinforcement must come after a behavior is emitted (a reinforcer is a consequence of the behavior).
21
Q
Shaping
A
• Shaping involves reinforcing successive approximations to the desired (target) behavior.
– Hot, warmer, cold
22
Q
Chaining
A
- A behavior chain is a connected (and usually ordered) sequence of distinct behaviors
- e.g., Cooking
- Once a chain has been learned, it can be considered one extended act.
23
Q
Chaining Methods
A
– 1st Step: Task Analysis—identify each link in the chain and their order
– Forward chaining begins with the first link in a task; when a link is mastered, the next link is added as a new requirement for reinforcement. And so on.
• If any link does not readily occur, use shaping.
– Backward chaining begins with the last link and works backward toward the first link.
24
Q
Prompting
A
Prompts are any stimuli that help an organism to initiate the desired behavior.
Some prompts are simply verbal or visual cues or reminders (sticky notes, or saying “Wash your hands.”)
Modeling (showing how) is another type of prompting.
Physical (or manual) prompting means physically guiding the organism to engage in the correct behavior.
e.g., If a dog simply will not spontaneously sit, push its rear end down, then reinforce.
25
Variables Affecting Reinforcement
* Contingency
* Contiguity
* Characteristics of Reinforcers
* Behavior Characteristics
* Motivating Operations
• Other odds and ends
26
Contingency
• Remember? If A, then B
* In operant conditioning, contingency refers to the degree of correlation between a behavior and its consequences.
* The rate of learning depends on the degree to which a behavior is followed by a reinforcer.
More contingency, more learning
* Smaller reinforcers with high contingency are generally better at producing learning than larger reinforcers with low contingency.
* Organisms will often work for a smaller “sure thing” than for a larger “long shot.”
27
Contiguity
* In operant conditioning, contiguity refers to the time between a behavior and its consequences.
* In general, the faster a reinforcer is delivered after a behavior, the stronger the learning.
Shorter delay, faster learning
• If a delay is too long, other behaviors might occur before reinforcement.
28
Characteristics of Reinforcers: Size
* All else being equal, larger reinforcers (e.g., more food) strengthen behavior more than smaller reinforcers.
* $1 vs. $100 after each A on report card.
* However, there are diminishing returns: as the size of the reinforcer increases, the rate of learning (while increasing) becomes less pronounced.
* i.e., a nonlinear relationship between size and learning.
29
Characteristics of Reinforcers: Quality
* Species, and individuals within a species, often have different “tastes.”
* e.g. Food is not interchangeable; some reinforce better than others.
* Rats prefer bread or milk over sunflower seeds: bread and milk strengthen behavior more than seeds.
* Reptiles do not respond to social reinforcers.
30
Behavior Characteristics
• Behavior that is a part of a normal repertoire for a species also affects reinforcement:
–Birds that peck for seeds in the wild (e.g., pigeons) learn to peck a disk for reinforcers much more readily than do birds of prey (e.g., hawks).
31
Motivating Operations
• Also called Establishing Operations
• MOs are anything that establishes conditions that improve the effectiveness of areinforcer.
–e.g. Thorndike’s work: animals were deprived of food to motivate them to find a way out of the puzzle box.
–Food deprivation established food as an effective reinforcer.
• Water (make thirsty)
• Social contact (keep isolated from others)
• Physical stimulation (sensory deprivation/boredom)
• Rest (make them tired)
• Warmth (keep in cold room)
32
Alternative reinforcers
• Reinforcement is weaker if there are alternative reinforcers easily available in the environment.
–A rat might not work hard for a particular type of food if there are other types lying around “for free.”
33
Extinction in Operant Conditioning
* In classical conditioning, if the CS (bell) is continually presented without the US (food), you get extinction: the CS will no longer elicit salivation.
* In operant conditioning, you find a similar process:
* If you withhold the reinforcing consequences of a behavior that has been reinforced in the past, the behavior will be weakened.
34
Another extinction phenomenon…
• When extinction begins, organism will begin to vary the previous behavior
–“try something different.”
• This can be useful in shaping: if the behavior gets “stuck” at one point, try withholding reinforcement.
• The organism will often begin variations in their behavior, perhaps offering a better approximation to the final target behavior.
35
Extinction and Emotion
* During the initial period of extinction, an organism will frequently increase its emotional behavior,especially aggression.
* Example: Rats who have been frequently reinforced with food for pressing a lever.
* Stop delivery of the food: Rats will become aggressive, biting at the lever, attacking other nearby rats.
36
Extinction-Induced Aggression
• Most organisms will attack when they are shocked, prodded, or subjected to other aversive stimuli.
• Apparently, removal of positive reinforcement is an aversive event.
– Extinction periods from previous reinforcement can been used as punishments to weaken other behaviors.
37
Spontaneous Recovery
* During extinction, it is typical for a behavior to continue decreasing until it reaches a pre-reinforced level or ultimately ceases.
* However, a phenomenon commonly associated with extinction is the reappearance of the behavior, after a delay,even though it has not be reinforced.
* This is called“spontaneous recovery,” just as in classical conditioning.
* Spontaneous recovery is short-lived and limited if the extinction procedure remains in effect.
38
Resurgence
* Condition Behavior A through reinforcement, then extinguish the behavior by withholding the reinforcement.
* Now condition Behavior B using the same reinforcer, then start to extinguish it.
* What happens?
* Subject engages in Behavior A again.
39
Resistance to Extinction
• The rate at which a behavior is extinguished by non-reinforcement depends on many factors.
• Continued responding during extinction is termed resistance to extinction.
• As a rule of thumb, any factor that strengthens reinforcement might also produce resistance to extinction.
• The number of times a behavior has been reinforced in the past affects resistance to extinction.
• A behavior with a long history of reinforcement will be more resistant to extinction than a behavior with fewer reinforcements.
– Number of bar presses during extinction varies by number of previous reinforcements.
40
Effort and Resistance to Extinction
• If a response requires greater effort, it may decrease more quickly during extinction than a response requiring less effort.
41
Guidelines for the Use of Extinction for Eliminating Unwanted Behavior
* Identify & withhold all sources of reinforcement for the unwanted target behavior.
* Withhold Consistently
* Consistency is important to all behavior change procedures; it is absolutely essential to extinction.
* Use Instructions (with children)
42
Guidelines for Extinction
| • Expect Common Extinction Phenomena:
–The behavior might get worse, not better, at first.
• Extinction bursts will occur
–The behavior might come back briefly.
• Spontaneous recovery will occur
–Aggressive and immature behavior might appear
• Resurgence and extinction-induced aggression
• These are indicators that extinction is working
–so don’t get dismayed, and stay with the plan.
43
Olds (1958)
* Using the electrodes again, Olds rigged a Skinner box so that depressing a lever automatically produced the mild stimulation of the brain.
* The rats were soon pressing the lever at a very fast rate, as many as 700 times in an hour.
* When allowed, rats would repeatedly depress the lever for over two days, until they collapsed from exhaustion.
* They would seldom eat, preferring the stimulation to food or water.
44
Getting Info from one neuron to another: Neurotransmitters
Neurotransmitters (chemicals) released from the sending neuron, travel across the synapse and bind to receptor sites on the receiving neuron, thereby influencing it to generate an action potential.
45
Dopamine
• Dopamine is released by almost everything that we experience as pleasurable— –Eating
–Winning a fight
–Sex
–Most addictive drugs, but cocaine produces especially high dopamine release.
–Dopamine is particularly high when a “reward” is unexpected.
–As rewards become more predictable, less dopamine is released (diminishing returns in reinforcement?)
46
Effects of blocking dopamine
• Wise et. al (1978): rats trained to routinely press a lever for food.
• After this, rats divided into 3 groups:
–Control + Continued Reinforcement
–Control + No Reinforcement
–Primozide+ Reinforcement
–Primozide is an effective dopamine blocker: blocks dopamine receptor sites.
47
Theories of Reinforcement
How does a reinforcer produce an increase in the probability of the reinforced response?
Two Approaches
• 1. Reinforcers are certain kinds of stimuli
–Drive Reduction Theory
• 2. Reinforcers are certain kinds of behaviors (responses)
–Relative Value Theory and the Premack Principle
–Behavioral Bliss Points & the Relative Deprivation Hypothesis
48
Drive Reduction Theory
• Clark Hull
– Organisms attempt to maintain a state of physiological equilibrium, or homeostasis
– Deviations from homeostasis produce biological drives that are aimed at bringing about homeostasis
• examples: hunger,thirst
– Reinforcers are stimuli that reduce biological drives
• examples: food to a hungry organism, water to a thirsty one
– Accounts for effectiveness of motivating operations (e.g. deprivation) and primary reinforcers(such as food) in learning.
– Also accounts for satiation: a well-fed rat will not be reinforced by food.
– Why? Its drive is already reduced
49
Reinforcers as Responses
• Some theories conceptualize a reinforcer as a response (not a type of stimulus).
– Food is not a reinforcer, the act of eating is.
• David Premack’s Relative Value Theory
• What makes eating a reinforcing response?
– Not a physiological drive state.
– A particular operant behavior (like eating) is a reinforcer if it has a high baseline probability of occurrence.
– We might say that high baseline behaviors are preferred behaviors, or have a high relative value
• Behaviors can be ranked by preferences: observe the whole repertoire of behaviors. Rank by probability of occurring (preference).
50
Premack Principle
• Organisms will work at a less preferred behavior for the opportunity to engage in a more preferred behavior.
– aka Grandma’s Law:
– Eat your spinach first (less preferred), then you can go out and play.
51
Behavioral Bliss Point
• The preferred frequency of a behavior (its relative value in Premack’s terms) is called the behavioral bliss point for that behavior
• With no restrictions, how much time would you spend:
– Working
– Studying
– Social networking
– Watching TV
– Housework
– The time you would spend on each activity is your bliss point for each activity.
52
The Response Deprivation Hypothesis
* The response deprivation hypothesis is that any behavior can be used as a reinforcer if the behavior is first constrained to be below your bliss point.
* Even initially very low probability behavior can be turned into a reinforcer if the behavior is constrained to be below your bliss point.
* How often would you like to scratch your nose?
* Scratching your nose is a low frequency behavior.
* But if your hands are tied, you are still below your (very low) bliss point.
* You might work really hard to be freed to scratch your nose.
* (working hard for “forbidden fruit”)
53
Schedules of Reinforcement
• A schedule of reinforcement is a set of rules for delivery of reinforcement
–Different schedules lead to different rates and patterns of behavior, which are called schedule effects.
–One distinction we can make is between continuous and intermittent (or partial) reinforcement schedules.
54
continuous reinforcement
(CR) schedule provides reinforcement every time a response occurs.
–CR produces very rapid learning, but it also yields the fastest extinction after the reinforcement is terminated.
–(Extinction can also be considered a schedule of reinforcement in which no behavior is reinforced.)
55
Schedules of Reinforcement
* Most schedules of reinforcement are intermittent(partial) rather than continuous. In other words, some responses are reinforced and others are not.
* This is more like the real world (you win some, you lose some).
* We can distinguish between two types of intermittent schedules:
– Ratio schedules: the delivery of reinforcement depends on the number of responses given by the individual.
• Fixed vs. Variable
– Interval schedules: delivery of reinforcement depends on the amount of time that has passed since the last reinforcement.
• Fixed vs. Variable
56
fixed-ratio schedule
provides reinforcement only after a definite (“fixed”) number of correct responses have been made.
– e.g., a laboratory rat being reinforced for hitting a lever after every 5 hits is being reinforced on an FR-5 schedule.
–The local coffee shop gives you a card that says if you buy 9 coffee drinks you will get the 10th beverage for free:
– FR-9
57
Fixed Ratio
• In FR schedules, there is high performance, but there is often a temporary pause in behavior immediately following reinforcement
. • This is called a post-reinforcement pause.
– The higher the order of the ratio (more work required before reinforcement), the longer the pause.
– The pause will be longer for an FR-100 schedule than for an FR-20 schedule.
58
Why the pause? (fixed ratio)
• Some have argued that fatigue causes the pause—after all that work, one gets tired.
• But some schedules produce even greater effort, yet there is no pause.
–The going explanation is that hard work is aversive, and we escape by avoiding the behavior for a while.
–Similar to procrastination—we put off studying not because we are tired, but because studying can be unpleasant.
59
After the pause… (fixed ratio
• After the post-reinforcement pause, regardless of its duration, the organism returns to a steady rate of responding.
–Called a “break-and-run” pattern –Take a break, but take off running hard afterward
• So an FR-100 schedule produces a longer pause than FR-20, but the rate of responding, once begun, is the same in both schedules.
60
variable-ratio schedule
provides reinforcement after a variable number of correct responses, working out to an average number in the long run.
• For example, in a VR-5 schedule, reinforcement sometimes follows a single response, sometimes after 8 responses, etc., but the average is about 5 responses.
• Fishing, golfing, “spare change?”
• Slot machines, like all gambling, provide a particularly compelling form of variable ratio reinforcement to the player
61
Variable Ratio
* Produces very rapid learning, as fast (or faster) than fixed ratio schedules.
* However, the post-reinforcement pause is eliminated (or greatly reduced in duration).
62
fixed-interval schedule
provides reinforcement for the first response made after a specific time interval (since the last reinforcement).
– A person who is paid every two weeks is reinforced for work on a fixed interval schedule (FI-2 weeks).
• i.e., if you are still working two weeks after your last paycheck, you will get another pay check
– If you receive your local newspaper at the same time every day, you are reinforced for looking on a fixed interval schedule (FI-24 hr)
• i.e., you look 24 hrs since you last got your paper.
63
Fixed Interval
–Immediately after reinforcement, there is a reduction in the rate of responding.
–But as the time interval for reinforcement approaches, there is very rapid responding until the reinforcement is delivered.
–Called a “scallop-shaped” pattern
–No matter how long the time interval, the organism will begin increased responding as the payoff time approaches.
– Checking the oven when baking (if the recipe says bake 10 minutes, you will not check after 2-3 minutes, but you will start checking frequently around 8-9 minutes)
– More studying as scheduled exams approach (to make sure you are reinforced with a good grade).
64
variable-interval schedule
provides reinforcement after a variable amount of time has elapsed since last reinforcement (but with an average time).
• If your newspaper delivery person is very inconsistent about delivery times, showing up one day at 5:00AM, the next day at 7:30AM, etc., but on average at 6:15AM, your paper is delivered on a variable interval schedule.
65
Variable Interval
* Rates comparable to (or sometimes higher than) fixed interval, but no scalloped effects.
* i.e., a smooth pattern of responding
66
Fixed Time Schedule
– Fixed time (FT) schedules: A reinforcer is delivered after a specific time regardless of behavior.
•In an FT-10s schedule, a rat gets food every 10 seconds, regardless of what it does.
– Can produce superstitious behavior: a behavior that happened to occur at the time of delivery might increase in frequency (although the reinforcement actually does not depend on the behavior).
» If a coach wears a hounds-tooth hat to a game and wins, he might begin wearing the “lucky” hat in the future.
» Occasional reinforcement (wins) might strengthen the behavior.
– Unemployment payments are similar to FT schedules if the check is provided every month regardless of the behavior of the recipient.
67
Stretching the Ratio
• Organisms can be trained to work very hard for very little payoff
– Rats can be reinforced to press a bar hundreds of times for a single reinforcer.
• This is accomplished by a form of shaping called “stretching the ratio.”
•Start with high reinforcement rate; slowly decrease (or “stretch”) the reinforcement rate.
•e.g., Begin with FR-3 schedule, then increase to FR-5, then FR-10, FR-20, FR-30, and so on, perhaps up to FR300.
68
Stretching the Ratio 2
• If the schedule is stretched too quickly or too far, signs of extinction may be observed
– Might see an extinction burst, followed by lower rate of responding.
– Might see aggression
– May see an increase in other responses
• This is known as ratio strain.
• If this happens, retreat to a denser (higher rate) reinforcement schedule.
69
Partial Reinforcement Effect
* PRE: Behaviors on intermittent (partial) reinforcement schedules are more resistant to extinction than behaviors on continuous reinforcement.
* PRE is counter intuitive: we hold on to learning better when reinforced less.
70
Concurrent Schedules
• In real life, operant conditioning is rarely a matter of being offered only one source of reinforcement.
• Instead, individuals typically must choose between alternative sources of reinforcement, and the reinforcements offered might require different types of behavior.
– Studyinggood grades
– Internetentertainment, social reinforcement
– Workpay, skill development
– Choices: What will you do most often?
71
Choice in the lab: Concurrent Schedules
* Two or more basic schedules are operating independently at the same time for two or more different behaviors
* The organism has a choice of behaviors and schedules.
72
Concurrent Schedules 2
•All else being equal, when different reinforcement schedules are provided for each of the concurrent responses:
– The response receiving the higher frequency of reinforcement will increase in rate
– The response requiring the least effort will increase in rate •Pressing a lever vs climbing a wall to press a lever
– The response providing the most immediate reinforcement(contiguity) will increase in rate
73
The Matching Law
• Behavior on concurrent schedules follows Robert Herrnstein’s Matching Law.
• Herrnstein has an equation to represent the matching law. For 2 behaviors reinforced on two schedules, the equation is
B1/(B1+B2) = R1/(R1+R2)
– The proportion of behavior devoted to a choice is the same as the proportion of reward offered.
74
Conger and Killeen (1974) Study of Matching in Humans
* They asked volunteers to participate with two others (confederates) in a discussion session on drug abuse.
* While the volunteer was talking, the two confederates sat on either side and intermittently expressed approval on different schedules.
* They found that the relative amount of time the volunteer looked at each confederate matched the relative frequency of verbal approval delivered by that confederate.
75
ALWAYS REMEMBER
Punishment always weakens a behavior; lowers the likelihood of response.
Positive punishment by adding a (usually aversive) stimulus.
Negative punishment by removing a (desired) stimulus.
76
Note on Negative Punishment
• Two types of negative punishment:
• Response cost: take away a specific reinforcer
– Money
– TV viewing
– Computer time
• Time Out: Remove all reinforcers For a while
– Keep it brief (1 minute per year of age)
– Make sure the time out location is not more reinforcing than the alternatives (“Go to your room” might be better than sitting at the dinner table.)
• With all punishment in humans, give an explanation of what is being punished.
– Sometimes kids don’t know exactly what they did to deserve punishment
77
Measuring the Effects of Punishment
• A widely used measure of the effects of punishment on a behavior is the called the suppression ratio.
• Suppression Ratio: a percentage of the responses that would be expected without the punishment.
– A suppression ratio of 90% would indicate a small drop in the rate of behavior (due to the punishment).
• i.e., the behavior is still occurring 90% of the time that it would occur without punishment.
– A suppression ratio of 30% would indicate a large drop in the rate of behavior
78
Does punishment work?
• Both E. L. Thorndike (Law of Effect) and B.F. Skinner said “no.”
• Both Skinner and Thorndike were influential researchers, so their dismissal of punishment as a way to control behavior was taken seriously for many years.
• Parents and teachers were discouraged from using punishment.
– Not because it was cruel, but because it was thought to be ineffective.
79
Skinner’s early work on punishment
Phase 1:
Rats were reinforced with food for bar pressing on a VI schedule until a high rate of pressing was obtained.
Phase 2:
On two successive days, rats were placed on extinction (no reinforcement regardless of behavior).
During the first 10 min of extinction on day 1, one group of rats was punished for each bar press (paw slapped); the other,control group,was not punished.
Rats then simply observed for rest of day 1 and day 2
80
Results of Skinner’s Experiment on Punishment
Punishment suppressed responding while it was being administered,but responding began to increase shortly afterward.
By the end of day 2, the punished rats ended up making as many responses overall during extinction as the unpunished controls.
81
Thorndike and Skinner were wrong about the effects of punishment
* Punishment has subsequently been shown to effectively suppress behavior, but many variables affect the strength of punishment.
* Contingency
* Contiguity
* Intensity
* Introductory Level of Punishment
* Nature of Previous Reinforcement
* Deprivation Level
82
Contingency (punishment)
• The effect of punishment in weakening a behavior depends on contingency between the behavior and the punishment. • If a rat is punished (e.g., shocked) every time it presses a lever, but not otherwise, then pressing will be less likely to occur.
83
Contingency (punishment) Boe and Church
• Rats trained to press a lever for food, then put on extinction for 20 min.
– During first 15 min.,
• Some rats (punishment group) were shocked only when they pressed the lever.
• Some rats were shocked at random, regardless of what they did (non-contingent shock group).
• Some rats (controls) were never shocked.
• Percentage of time pressing the lever was recorded for all three groups.
84
Contingency (punishment) Boe and Church (results)
• Boe & Church (1967)
– The rats who were given non-contingent shocks did not differ from the control rats who were under extinction.
– The punished rats had dramatically lower rates of responding than both of the other groups.
• So, punishment is effective when it consistently follows the behavior.
85
Contiguity (punishment)
| Camp et al., 1967
• The time interval between a behavior and a punishing consequence has a strong effect on the rate of responding.
• Camp et al. (1967)
– Rats were periodically reinforced for pressing a lever, but were sometimes shocked for the same behavior.
• Some shocked immediately (0 second delay)
• Some shocked after 2 seconds delay
• Some after 30 seconds delay
• Suppression ratios calculated for the 3 conditions.
86
Contiguity (Camp et al., 1967)
• Suppression Ratio measured for all three groups.
– For the immediate shock group, pressing dropped to less than 10% of expected responding.
– For the 2 sec delay condition, it was between 1020% of expected.
– For 30 sec delay, pressing was around 40% of expected.
• So, punishment is more effective when it immediately follows the behavior.
87
Intensity of Punishment
• In general, the stronger the punishment (more aversive), the faster the behavior will decline.
• Very mild punishments have little effect on behavior.
– Thorndike’s “wrong.”
– Skinner’s “slap on the wrist”
• Much research shows that stronger punishments
(e.g., greater voltage of shocks) can greatly suppress behaviors.
88
Introductory Level of Punishment 1
• How can you find the right intensity of punishment that will suppress a behavior?
• You could start with strong punishment and gradually weaken it, or
• You could start with mild punishment and gradually increase the intensity.
– This question has implications for strategies for real life punishment: cheating policies, tax evasion, etc.
• Should you start small and let possible consequences “sink in”?
• Or should you hit ’emhard first time so that don’t forget it?
89
Introductory Level of Punishment 2
• Research shows:
– If you start with weak punishment, organisms tend to continue responding even as the intensity of punishment increases.
– Often punishments that normally would be successful if they had been applied from the start will not work if preceded by weak punishments.
90
Introductory Level of Punishment 3
* Real life punishments often follow this weak-at first pattern.
* Traffic violations: I’ll just give you a warning this time. Next time a ticket. Next time suspended license.
* Same with children: stern look; scolding; swat on the backside; beat the tar out of them.
91
Nature of Previous Reinforcement
• The frequency, amount, and quality of reinforcement history will affect punishment in obvious ways.
– Behaviors will persist if there was, or continues to be, a “good source” of reinforcement available for the behavior.
• Behavior will also persist if there is no other source of reinforcement.
– A rat will continue to press a lever, in spite of shocks, if food can’t be obtained any other way.
92
Deprivation Level
• Organisms that are deprived of a primary reinforcer (food or water) will be less influenced by punishment than a satiated organism.
– Hungrier rats will press a lever longer in the face of shocks.
• Azrinet al. (1963)
—varied weight of rats with food deprivation (60-85% normal weight).
• Observed rate of lever pressing for food when pressing was also accompanied by shock.
93
Theories of Punishment
| just the names
* Disruption Theory
* Two-process Theory
* One-process Theory
94
Disruption Theory
* Skinner and Guthrie (and others) thought that instinctive, often emotional, reactions to aversive events were simply incompatible with the reinforced behavior.
* A rat will jump, freeze, or even run when shocked.
* These behaviors interfere with steady lever pressing.
* This is sometimes called response competition: an instinctive response is competing with an operant response.
* Therefore lever pressing will temporarily decline during punishment.
95
Problems with Disruption Theory
* Response competition alone is insufficient to make punishment effective.
* When punishment is contingent on behavior instead of just co-occurring at random, it is much more effective.
* Random shocks do not produce the same punishing effects of contingent shocks, even when the shocks occur with the same frequency.
96
Two Process Theory
• Two-process theory states that punishment involves both classical (Pavlovian) conditioning and operant conditioning.
– When a rat is shocked for pressing a lever, the lever becomes a conditioned stimulus for pain and fear.
• Lever : Shock = fear/pain
– As a result, the lever becomes an aversive CS, and the rat will avoid it, thus decreasing the rate of lever pressing.
– Avoidance (of the feared lever) is then negatively reinforced.
97
Two Process Theory 2
* This is a popular theory, but there are inconsistent research findings.
* In some studies, organisms no longer show fear of the lever, but continue to stop pressing anyway.
* Also: Although a rat might stop pressing a lever, it does remain exposed to it to some degree.
* Over time, with no shocks, fear of lever should extinguish.
* When fear is extinguished, the rat should start pressing again.
* But when shocked, the fear and avoidance return.
* Should see an oscillating pattern of responding over time.
* This is not observed.
98
One Process Theory
– Punishment does not involve classical conditioning, only operant conditioning.
• This was Thorndike’s original view, before he decided that punishment actually had no effect on behavior.
– Originally, he argued that punishment was simply the mirror image of reinforcement.
– His theory was that reinforcement strengthens a neural connection between a behavior and an event.
– Punishment weakens a neural connection between a behavior and an event.
– This was called the “Negative Law of Effect”
99
One Process Theory 2
• Problem with Thorndikes “neural weakening” theory:
– Skinner’s finding that suppression of behavior is sometimes only temporary contradicts this.
– Why would a weakened neural bond regain its strength over time under conditions of extinction?
100
One Process Theory
* Supporters of one-process theory today disregard neurological speculation, adopting relative value theory.
* They claim that if punishment is just a mirror image of reinforcement, then the Premack Principle should apply to punishment as well as reinforcement.
* Original Premack: Eat your spinach, then you can eat your cake.
101
Premack and Punishment
* If punishment is just another form of operant conditioning, then a less preferred behavior should be able to punish a more preferred behavior.
* Premack for Punishment: If you eat your cake, I’ll make you eat a plate of spinach.
* Much research confirms that the Premack Principle applies to punishment as well as to reinforcement.
* Mazur (1975):Hungry rats will reduce the rate pressing a lever for food (a preferred behavior) if the rat is also made to run in a wheel (less preferred) after eating.
102
Problems with Punishment
• Some common side effects of punishment:
• Punishment is aversive and threatening, so it can lead to fight or flight responses.
• Flight: Escape/Avoidance
• People and other organisms will simply avoid the punisher or the punishing situation.
• Related: Conditioned fear of punisher
• Do you want kids to always be afraid of you?
• Fight: Aggression (as with extinction)
– Attacking, revenge, etc.
– After being shocked, rats will attack other rats and objects.
103
More Problems with Punishment
• Apathy: Sometimes an organism will simply stop behaving (period) if punishment is severe enough.
– Seligman’s learned helplessness theory of depression.
• Imitation of Punisher (via observational learning):
– Punishment can become a tradition.
– Punitive managers have subordinates who become punishers.
104
Benefits of Punishment
• If punishment is immediate, consistent, and strong enough, it actually works.
– This can be useful with seriously destructive behaviors, such as assault or self harm (head banging, scratching face)
– Downside: because it is effective, it might be overused.
• Severe spanking will stop misbehavior, so parents might use severe spanking for everything. (Abuse)
105
Other benefits of punishment
• Sometimes leads to an increase in pro social behavior as a side effect (not just decrease in antisocial behavior)
–Being nicer, friendlier, apologizing, etc
–This might reflect innate appeasement behaviors to help repair and restore relationships
106
Alternatives to Punishment
• Response Prevention
– For seriously harmful actions, physical restraint might be necessary
• Remove any eliciting stimuli associated with misbehavior
– Sometimes misbehavior occurs only in certain situations or in the presence of specific stimuli.
– How does your son behave around different friends?
– If he misbehaves only when he’s hanging out with Johnny, keep him away from Johnny.
107
Differential Reinforcement of Alternative Behaviors (DRA).
– Combines extinction of unwanted behavior with reinforcement of other behaviors
– So, you don’t punish the unwanted behavior, you simply don’t reinforce it (e.g., ignore it).
– But you also reinforce more appropriate behaviors.
– DRI–differential reinforcement of incompatible behavior
• Reinforce alternative behaviors that are incompatible with the unwanted behavior.
• Rather than punish running around the room in class, strongly reinforce sitting still
learning and behavior Berry
flashcards
Decks in class (3)
# Cards
