what is the traditional view of perceptual processes involved in language comprehension?
meanings are represented as features, usually described in terms of other words - like a dictionary
meanings need to be grounded in the physical, social and emotional world - symbol grounding problem, Chinese dictionary example (continuously looking up a Chinese word you don’t understand but that definition is in Chinese)
much of the learning of words consists in relating them to objects and actions, which manifestations in the real world and are accessible through perception and action
what is the embodied view of perceptual processes involved in language comprehension?
Barsalou (1999)
represent language using same systems as would use to represent perception and action
when learning language, words occur with objects association with because in everyday life, words often co-occur with objects they refer to, these words get associated with the memory traces that stem from interacting with those objects
experiential simulations view of comprehension holds that comprehenders mentally simulate the described objects, states and events during comprehension
what are perceptual representations in the embodied view?
when understanding the meaning of a word, we create a perceptual simulation or mental image of the object that the word describes
visual orientation, object shape, visibility
how would the traditional view say people mentally represent “the pencil is in the cup” (Stanfield & Zwaan, 2001)?
human mind manipulates abstract, arbitrary and amodal symbols
this combination of manipulation and symbolic representation constituted as cognition
language represented in terms of symbols and rules like a computer
[IN[PENCIL,CUP]]
how would the embodied view say people mentally represent “the pencil is in the cup” (Stanfield & Zwaan, 2001)?
participants should mentally represent the orientation of the pencil based on their experiences
what did the sentence-picture verification task show about how people mentally represent “the pencil is in the cup” (Stanfield & Zwaan, 2001)?
read sentence - John put pencil in the cup - then presented with a paciture of the object
half the time, picture matched orientation implied by sentence and half the time it didn’t
time taken to recognise pictures of the object in each orientation measured
participants faster to respond and say that object had been mentioned in the sentence when it matched orientation which suggests readers had mentally simulated orientation of object when reading sentence
what was Zwaan, Stanfield & Yaxley’s (2002) study into object shape?
sentence-picture verification
read a sentence (“Mary saw eff in the eggbox”) then presented with picture of egg in shell or fried egg
was item mentioned in previous sentence?
participants faster to indicate object had been mentioned in the condition where it matched object shape implied by the sentence than the condition in which it did not
taken as evidence that participants had mentally represented the shape of the object
what was Yaxley & Zwaan’s (2007) study into visibility?
through clean goggles, skier could easily identify moose but through fogged goggles, skier could hardly identify moose
presented with pictures that either matched or didn’t match the implied visibility
faster when visibility matched which suggests degree of visibility implied in linguistic context can influence object recognition
simulation of linguistic descriptions not limited to activation of intrinsic object properties but also invokes perceptibility of objects implied by environmental context
when reading, creating perceptual representations of environments
what was Wassenburg & Zwaan’s (2010) eye tracking study?
three phases presented as unrelated experiment
phase 1 - word-picture verification task - does picture match word? either presented with horizontal or vertical, put certain mental image in person’s head
phase 2 - filler task - mental rotation task, 15-20 minutes, to make it less likely that they would be able to guess the aims of the experiment
phase 3 - eye-tracking experiment - while read piece of text that talked about object, increased reading time when mismatched mental image, previous exposure to a picture of an object in a particular orientation affected reading times for phrases that described that particular object
what was Coppens, Gootjes & Zwaan’s (2012) ERP study?
three studies presented as unrelated experiments
experiment 1 - word-picture verification task - does the picture match the word? ironing board either folded up away or folded out to be used
experiment 2 - filler task - emotional word Stroop task, 15 minutes
experiment 3 - sentences were presented word-by-word while EEG signals were being recorded, contained object that either matched or mismatched particular shape of object, when participants in mismatch condition higher spike in electrical brain activity because perceptual images matched or didn’t
what is the traditional view of motor processes involved in language comprehension?
word meaning processed in language areas of brain
what was Hauk, Johnsrude & Pullvermüller’s (2004) fMRI study?
are same part of the brain for performing actions activated for when saying words describing actions? whether action words would also activate motor areas of the brain
completed “localiser” task where actually performed with feet, fingers or tongue while in MRI scanner so researchers could localise areas of brain that were activated when participants actually performed actions
then performed passive reading task where presented with words referring to face, arm or leg actions
overlap in brain regions - simulation of actions going on
words and actions that “fire together, wire together” - word meanings become grounded in action
what was Raposo et al’s (2009) study into action words in context?
performed localiser task while performing actions
presented with either a sentence where actual action performed or presented with sentence where same action word was used but idiomatically
found when sentences describing literal action is overlap in brain area
when reading sentences that doesn’t actually mean action, no overlap
no activation in motor areas for idiomatic sentences
motor responses context-dependent, not automatic and invariable
what are the criticisms of Raposo et al’s (2009) study?
fMRI data based on blood oxygenation levels in particular brain region
levels of blood oxygenation slow to respond to cognitive events so any early or short-lived activations may not be picked up
what is the action-sentence compatibility effect?
read sentence describing directional hand movement
when reading sentence, hold down middle button
half participants had to say yes by pushing button far away and half had to push button near - had to say if sentence grammatical
participants faster to move hands in the direction implied in the sentence
effects arise because people are simulating a “toward” or “away” movement and actual movement of fingers either is compatible, leading to a faster response, or incompatible, leading to a slower response
similar effects observed for “abstract” transactions
direction of information transmission is important but no actual bodily movement is implied
do action-sentence compatibility effects replicate?
Morey et al (2022)
attempt to replicate action-sentence compatibility effects across 18 labs
used adapted version of Glenberg & Kaschak’s (2002) task
effects failed to replicate across all 18 attempts
what was Zwaan, Taylor & de Boer’s (2010) study into the action-sentence compatibility effect?
participants made a story appear phrase-by-phrase by turning a dial
half participants had to turn clockwise, other half anticlockwise
in half critical sentences, description of action being carried out that required manual rotation action and half depicted someone intending to act in such ways
mismatch effects found for current actions
no mismatch effects found for intended actions
what is the body specificity hypothesis (Casasanto, 2009)?
according to theories of embodied cognition, thoughts comprise mental simulations of bodily experiences
if this is true, people with different kinds of bodies must represent language differently and people with different bodily characteristics, who interact with their physical environments in systematically different ways, should form corresponding different mental representations
what is the handedness effects?
if thinking about actions involved mentally simulating the way we typically execute them, actions that we perform with our dominant hands should result in different patterns of brain activity in right-handed and light-handed people
it does
similar activity observed when reading action verbs
how are linguistic associations experientially simulated?
phrases like “the right answer” associate good things with rightward space
“out in left field” and “two left feet” associate bad things with leftward space
Latin words for right (dexter) and left (sinister) form the roots of English words meaning “skilful” and “evil” respectively
what are body specific associations?
most people have a dominant hand, usually the right hand, and therefore interact with their environment more fluently on one side of body-centred space than the other
people come to implicitly associate good things with the side of space they can interact with more fluently and bad things with the side of space they interact with less fluently
since most people are right-handed, this would result in a bias towards good things being associated with the right and bad things with the left
what was Holt & Beilock’s (2006) study into action and expertise?
ice hockey experts and novices read sentences describing hockey and non-hockey situations
used sentence-picture verification task
was target mentioned in previous sentence
both should get mismatch with balloon but only hockey experts should get mismatch with hockey helmet condition
results for non-hockey items showed all participants responded more quickly to objects that matched the action implied in the sentence than to those that did not
for hockey items, results showed that experts responded more quickly to matches than to mismatches but novices didn’t
what are the challenges for embodiment?
if you believe it, how do you uncover the nature of the simulations that people produce?
if you don’t believe it, how do you provide evidence for “no simulation”?
much of existing evidence demonstrates the effect of language on picture recognition/bodily movement rather than other way around
what about abstract sentences?
what are the things to consider in embodiment?
how exactly do simulations work in relation to comprehension?
what do simulations actually consist of?
how do their real-time characteristics relate to the time it takes to comprehend a sentence?
are these questions really answered by paradigms such as the action-sentence compatibility effect?
what if some of the key findings don’t replicate?
-
introduction to perception46
-
the visual pathway 155
-
the visual pathway 245
-
colour perception47
-
beyond v121
-
object recognition40
-
face perception30
-
false and recovered memories32
-
encoding failures26
-
effective encoding29
-
storage failures25
-
active forgetting23
-
types of retrieval22
-
improving retrieval22
-
autobiographical memory21
-
visual word recognition 133
-
visual word recognition 267
-
embodiment24
-
shallow processing29
-
perception and bayesian inference35
-
bounded rationality35
-
heuristics27
-
attention and consciousness39
