1 Describe Thorndike’s puzzle box apparatus and his use of it to study the behaviour of cats (and other animals). Why did he study animal behaviour? What were his findings? What conclusions did he draw from those findings? (pp. 127–130)
Thorndike placed cats in a puzzle box where they had to perform specific actions (like pulling a lever) to escape and obtain food. He studied animals to understand basic learning processes without relying on introspection. Over repeated trials, escape times decreased gradually.
He concluded that learning is incremental and based on consequences, not insight. Behaviours followed by satisfying outcomes are strengthened, leading to the formulation of the law of effect.
2 Define Thorndike’s law of effect. What did Thorndike speculate about reinforcement’s neural effect? What is this view called? (p. 130)
The law of effect states that responses followed by satisfying consequences are strengthened, while those followed by discomfort are weakened.
Thorndike speculated that reinforcement strengthens neural connections between stimulus and response, known as the connectionist view of learning.
3 Describe how the capacity to benefit from the consequences of one’s own actions contributes to survival. Illustrate by referring to Metzgar’s (1967) experiment. (p. 131)
Learning from consequences allows organisms to adapt behavior to maximize rewards and avoid danger, improving survival.
Metzgar showed that animals that learned to avoid predators based on experience had higher survival rates, demonstrating the adaptive value of consequence-based learning.
4 Describe the essential components of a Skinner box. How did the Skinner box get its name? (p. 132)
A Skinner box includes a chamber, a manipulandum (lever/key), a reinforcer delivery system (like food), and often stimulus controls like lights or sounds.
It is named after B.F. Skinner, who developed and used the apparatus to study operant conditioning systematically.
5 What is operant learning or operant conditioning? What other name refers to operant conditioning? (pp. 131–133)
Operant conditioning is learning in which behavior is shaped and maintained by its consequences, such as reinforcement or punishment.
It is also called instrumental learning, emphasizing that behavior serves as a tool to produce outcomes.
6 Define reinforcement. Name the three essential features of reinforcement. Be able to explain and reproduce the contingency square (Figure 5-5). (pp. 133–134)
Reinforcement is a process in which a consequence increases the likelihood of a behavior.
Its three features are: (1) a response occurs, (2) a consequence follows, and (3) the response increases in frequency. The contingency square shows positive/negative reinforcement and punishment based on adding or removing stimuli.
7 What are the two basic types of reinforcement? What are their similarities and differences? Be able to provide and recognize original examples of each type. (pp. 133–134)
Positive reinforcement involves adding a stimulus to strengthen behavior, while negative reinforcement involves removing an aversive stimulus.
Both increase behavior, but differ in whether something is added or removed. Example: giving praise (positive) vs. stopping loud noise when a button is pressed (negative).
8 What is escape training? (p. 134) Comment: The text does not make a clear distinction between escape and avoidance. In escape, an organism’s response terminates an aversive stimulus. For example, on a sunny day, putting on a pair of sunglasses (a response) terminates the glare of the sun (an aversive stimulus) that you (the organism) are experiencing. In avoidance, the response prevents or postpones a consequence. For example, suppose you put on your sunglasses while still inside your house. Putting on the sunglasses does not terminate the glaring sun consequence, but it does prevent it from occurring (avoidance). In escape, the response terminates some aversive stimulus. In avoidance, the response produces no immediate consequence. Instead, in avoidance, failing to respond produces a consequence. In our sunglasses example, when you put on the sunglasses while still inside, it has no consequence: it does not produce or remove the sun’s glare. However, failure to put on the sunglasses (while still inside) will lead to the aversive glare of the sun when you step outside. Another example of an avoidance contingency is the late charge you are assessed on your utility bill. If you pay your bill on time, nothing happens: you do not receive a late charge. However, failure to pay your bill on time produces the aversive late charge. In a conceptual exercise later in this unit, you will practise discriminating between instances of escape and avoidance.
Escape training occurs when a response terminates an ongoing aversive stimulus, strengthening that behavior.
As noted, escape differs from avoidance: escape ends something unpleasant already happening, while avoidance prevents it from occurring, often with no immediate consequence.
9 Describe some of the potential problems in differentiating positive and negative reinforcement. Refer to the experiment of Hundt et al. (pp. 134–135)
It can be difficult to distinguish whether behavior is strengthened by adding a positive stimulus or removing an aversive one, especially when both occur together.
Hundt et al. showed that situations often involve mixed contingencies, making classification ambiguous and suggesting the distinction is sometimes more theoretical than practical.
10 Some things that would seem to be distinctly unrewarding can strengthen behaviour. Discuss and provide examples of this. (pp. 133–134)
Stimuli that seem unpleasant can still reinforce behavior if they function to reduce discomfort or serve other reinforcing roles.
Examples include engaging in effortful exercise for long-term benefits or completing difficult tasks to relieve anxiety, showing reinforcement depends on function, not appearance.
11 How does reinforcement of a response give that response momentum? Explain Nevin’s use of the metaphor of momentum to describe the effects of reinforcement. (p. 135) Comment: Everyday observations may help you confirm the usefulness of the metaphor of momentum in describing behaviour. For example, children have a hard time tearing themselves away from video games or television to do something else.
Reinforcement increases the persistence of behavior, making it resistant to disruption, similar to how physical momentum resists change.
Nevin’s metaphor suggests that frequently reinforced behaviors continue even when conditions change, as seen when children persist in playing games despite interruptions.
12 How did Page and Neuringer show that randomness is a reinforceable property of behaviour? (pp. 135–136)
They reinforced animals for producing variable, unpredictable sequences of responses rather than repetitive ones.
Results showed animals increased randomness when it was reinforced, demonstrating variability itself can be shaped by reinforcement.
13 What are primary (unconditioned) reinforcers? Discuss some of the less obvious reinforcers described in the text. (pp. 138–139)
Primary reinforcers are stimuli that are inherently reinforcing due to biological needs, such as food, water, and warmth.
Less obvious examples include sensory stimulation or novelty, which can function as reinforcers even without direct survival value.
14 What are secondary (conditioned) reinforcers? (pp. 139–140) Comment: It is common to refer to primary reinforcers as unconditioned reinforcers and to secondary reinforcers as conditioned reinforcers. Chance neglects to mention that primary reinforcers are also called unconditioned reinforcers.
Secondary reinforcers are stimuli that acquire reinforcing properties through association with primary reinforcers.
They include money, praise, or tokens, and are also called conditioned reinforcers, while primary reinforcers are unconditioned.
15 What four advantages do conditioned (secondary) reinforcers have over unconditioned (or primary) reinforcers? What key disadvantage do conditioned reinforcers have? (pp. 139–140)
Advantages: they are more flexible, can be delivered immediately, do not depend on deprivation states, and can be generalized across situations.
Disadvantage: they depend on learning history and can lose effectiveness if not maintained by pairing with primary reinforcers.
16 What are generalized reinforcers? (p. 140)
Generalized reinforcers are conditioned reinforcers associated with multiple primary reinforcers.
Because of this, they are less susceptible to satiation and remain effective across different motivational states.
17 What are contrived reinforcers? Why is it sometimes difficult to distinguish natural and contrived reinforcers (p. 141)?
Contrived reinforcers are artificially arranged consequences used to modify behavior, such as rewards in structured settings.
The distinction is difficult because natural environments often include arranged contingencies, and what counts as “natural” can depend on context.
18 How does operant conditioning differ from Pavlovian conditioning? (pp. 141–142)
Operant conditioning involves learning relationships between behavior and consequences, with behavior emitted voluntarily.
Pavlovian conditioning involves associations between stimuli, where responses are elicited automatically without control over outcomes.
19 Describe the concept of contingency in reinforcement. How and why does it influence reinforcement? (pp. 142–144)
Contingency refers to the dependency between a behavior and its consequence—reinforcement occurs only if the behavior happens.
Stronger contingencies produce stronger learning because they clearly signal that the behavior controls the outcome.
20 Describe the concept of contiguity in reinforcement. How and why does it influence reinforcement? (pp. 144–146) Comment: The author often discusses contiguity in terms of delay of reinforcement. The general principle of contiguity is that immediate reinforcement is more effective than delayed reinforcement.
Contiguity refers to the closeness in time between a behavior and its consequence.
Immediate reinforcement strengthens behavior more effectively because it creates a clearer association, while delays weaken learning due to interference.
21 How do the following conditions affect the effectiveness of a reinforcer? (a) size of the reinforcer, (b) task characteristics, (c) competing contingencies (pp. 146–149)
Larger reinforcers generally produce stronger effects, though diminishing returns may occur.
Task difficulty influences reinforcement effectiveness, and competing contingencies can reduce its impact by reinforcing alternative behaviors.
22 What is meant by motivating operations? Describe the two types and identify examples of each. (pp. 148–149).
Motivating operations are conditions that alter the value of a reinforcer and the likelihood of behaviors that produce it.
Establishing operations increase value (e.g., hunger increases food value), while abolishing operations decrease value (e.g., satiation reduces it).
23 Describe the work of Olds et al. (1954) and Talwar et al. (2002) on the electrical stimulation of the brain (ESB). What are the reward centres of the brain? (pp. 150–151)
Olds showed animals would repeatedly press a lever to receive brain stimulation, indicating powerful reinforcement. Talwar demonstrated similar effects in more naturalistic settings.
Reward centres include areas like the hypothalamus and nucleus accumbens, involved in pleasure and reinforcement processing.
24 Describe the role of dopamine in reinforcement. How does the research of Hollerman et al. (1998) seem to support the Rescorla-Wagner model (see Unit 2)? (pp. 151–153)
Dopamine signals reward prediction and reinforcement learning, increasing when outcomes are better than expected.
Hollerman et al. showed dopamine activity reflects prediction errors, aligning with the Rescorla-Wagner model’s emphasis on expectation and surprise.
