9>multimodality

Question 1

3 main types of modality>

Answer 1

• verbal
• vocal
• kinesic

Question 2

verbal modality=

Answer 2

language we refer to as words, sentences, structure

Question 3

vocal modality=

Answer 3

consists of 1>phonological prosody & 2>paralinguistic prosody

Question 4

vocal modality> phonological prosody=

Answer 4

(ORGANISATION of segments)
>e.g. information structure, phrase boundary segmentation, TCUs

Question 5

vocal modality> paralingusitic prosody>

Answer 5

less tied to lang
>e.g. temp, intensity, duration, voice quality, loudness

Question 6

kinesic modality=

Answer 6

includes gaze, posture, proxemics (interpersonal distancing), gesture, facial expression

Question 7

2 examples of the importance of nonvebral modalities >

Answer 7

• Gestures can carry about 50-70% of the message the speaker is encoding
• iconic hand gestures carry meaning

Question 8

Did the evolution of the 3 communicative modalities evolve in parallel?>

Answer 8

no
- verbal modality as last addition to system (between around 1.4 ma & ca 600,000 years ago)
- lang gradually co-evolved with pre-exisitng gestural & vocal modes of communication

Question 9

Redundant cross-modal patterning=

Answer 9

information in one modality being AMPLIFIED by information in another modality
(“what the hell’s going on here” (verbal) & use of pitch (vocal) & open hands (kinesic)

Question 10

contrastive cross-modal patterning=

Answer 10

information in one modality is MODIFIED or CONTRADICTED in another modality

(e.g. “pleased to meet you” (verbal) is contradicted by averted gaze, unsmiling (kinesic)

Question 11

What effect does the ‘micro-context’ provided by nonverbal choices have?

Answer 11

• it contributes to realising speech act &helps maintain, protect or attack faces

Question 12

examples of how modalities are intertwined>

Answer 12

-deaf signers use vocalisation & mouth movements
- speakers gesture frequently even when not visibly accessible to addressee

Question 13

What is gaze used for?

Answer 13

• used as part of ‘regularised contingencies of interaction’:
  >gaze for next speaker selection
  >mutual gaze as prevalent in assessment sequences
  >gaze for agreement

Question 14

problem with dyadic structure of dialogue>

Answer 14

• primarily ‘dyadic structure’ of dialogue (made for 2) poses constant danger of ‘marginalisation’
• marginalised conversationalists may then withdraw from dyad

Question 15

When is gaze aversion used?>

Answer 15

• to establish no longer participating (if been marginalised)
• if disengaged