1
Q
What are types of qualitative studies?
A
- Interaction with the user (observation, diary study, ..)
- Location of data collection (field, lab)
- Number of participants (individual, group)
- Modality of interaction (written, mediated oral, ..)
- Freedom in interaction (unstructured, semi-structured, fully structured)
2
Q
What is an individual interview?
A
- A talk with one user
- Face-to-tace, phone, video, conference, IM
- Open, semi-structured, structured questions
3
Q
When do you use an individual interview?
A
- Discover attitudes, beliefs, desires, and reactions towards a concept
- Context for quantitative measures
4
Q
What is a contextual interview?
A
- Watch people’s behavior in their own environment doing their own tasks
- Ask questions while observing
- Can be combines with scenarios/usability tests
5
Q
When do you use a contextual interview?
A
When context is important
6
Q
What are advantages of a contextual interview?
A
- Natural behavior
- Real life tasks/environemnt
- Explicit & observable knowledge
7
Q
Wat are disadvantages of a contextual interview?
A
- Costly
- Poor control over conditions
8
Q
What is laddering?
A
An interview technique that helps discover links between:
- Attributes of your product
- Consequences of this attribute (benefits, impact)
- Values that expose the cause behind someone’s interest in the product
9
Q
When do you use laddering?
A
Evaluate meaningfulness
10
Q
How does laddering work?
A
Keep asking why
11
Q
What are advantages of laddering?
A
Getting to high-level goals and underlying values
12
Q
What are disadvantages of laddering?
A
- Can feel tedious/childish
- Not easy to conduct
13
Q
What is a focus group?
A
An informal, semi-structured, moderated group discussion (n=5-10)
14
Q
When do you use a focus group?`
A
Discover attitudes, beliefs desires and reactions towards a concept
15
Q
What are advantages of focus groups?
A
- Cost effective
- Discover what users need, want, feel, value, ..
16
Q
What are disadvantages of focus groups?
A
- Explicit knowledge only (say, no do/use)
- Poor method for usability evaluation
- Requires advanced moderation skills
- Group dynamics
17
Q
What is think aloud?
A
User verbalize thinking process while performing tasks
18
Q
For what is think aloud for?
A
- Discover what users really think
- Uncover design flaws
19
Q
Wat are advantages of think aloud?
A
- Cost effective (low n sufficient)
- Flexible (prototypes and finished products)
- ‘Real time: measure of experience
20
Q
What are disadvantages of think aloud?
A
- Unnatural for users
- Chance of bias
- Individual differences
- Affects other measures
21
Q
What are critical tasks for the tink aloud protocol?
A
- Several minutes to complete
- Relevant for the product
- Challenging enough
22
Q
What are two strategies of think aloud?
A
- Rigid
2. Speech clues
23
Q
What is rigid with think aloud?
A
No interference, neutral reminders only (“keep talking”)
24
Q
What is speech clues with think aloud?
A
- Acknowledging contributions (“mm-hmm”)
- Ask for clarification
- Encouragement
25
What are observations?
- Observing users interacting with a product
- Fly on the wall
- Recordings
26
When do you use observations?
Understand how users use a product
27
What is controlled observation?
Observing users interacting with a product in a lab environment
28
When to use it?
- Specific goals/tasks
| - Context for quantitative measures (usability issues)
29
What are advantages of controlled observation?
- Good control over conditions
- Structured approach
- Quick
30
What are disadvantages of controlled observation?
- Pre-defined tasks only
| - Unnatural context / behavior
31
What is naturalistic observation?
Observing users interacting with a product "in the wild"
32
When do you use naturalistic observation?
When context is important
33
What are advantages of naturalistic observation?
- Ecological validity
| - Observe how product is used in daily life
34
What are disadvantages of naturalistic observation?
- Costly
- Little control over conditions
- Hawthorne effect
35
What is the Hawthorne effect?
The Hawthorne effect refers to a type of reactivity in which individuals modify an aspect of their behavior in response to their awareness of being observed.
36
What are aspects you can observe?
- Space
- Actor
- Activity
- Object
- Act
- Event
- Time
- Goal
- Feelings
- Reflection
- Interactoins
- Use in context
37
What is an expert review?
- Inspection method: analysis by UX expert
- Comparing against heuristics
- Holistic approach
38
When do you use an expert interview?
Uncover usability problems and strengths
39
What are advantages of expert interviews?
- Flexible
| - Quick
40
What are disadvantages of expert interview?
- An experts is not your average user
- Ideally multiple experts (=costly)
- Focus mostly on minor usability issues
41
What is a cognitive walkthrough?
- Is a usability evaluation method in which one or more evaluators work through a series of tasks and ask a set of questions from the perspective of the user.
- Task-driven instead of holistic
42
What are the steps in the cognitive walkthrough?
1. Define representative tasks (from the user's perspective)
2. Break these down into a sequence of actions
3. Follow each action and ask yourself:
a) Will users want to product whatever effect the action has?
b) Will users see the (UI) controls?
c) Will users know that the control produces the effect they want?
d) After the action is taken, will users understand the feedback?
43
What is the overview of qualitative data analysis?
1. Manual coding (computer aided)
- Data driven coding (bottom-up)
- Theoraticallhy driven coding (top down)
- Combination of top-down and bottom-up
a) qualitative report
b) quantitative report
2. Computerized analysis
- Sentiment analysis
a) quantitative report
44
What is sentiment analysis?
- Detect the polarity (positive or negative) of an opinion with a certain degree of confidence
- Uncover themes of sentiment in large scale data (e.g. twitter posts)
45
What is LIWC?
Linguistic Inquiry of Word Count
| - Dictionary-based text analysis for many emotional and cognitive categories
46
What are the advantages and disadvantages of LIWC?
+ Easy to use
| - Cannot handle negotiation (I am not happy), strictly rule-based
47
What does R package do?
Classifies sentences
48
What are the different classifications of R-package?
1. Positive
2. Negative
3. Very positive
4. Very negative
5. Neutral
49
What is coding in manual analysis of qualitative data?
Systematically code your data for reoccurring issues/themes and sub-issues/sub-themes
50
What is the goal of coding?
Reduce data into meaningful categories and sub-categories in a systematic way, finding patterns
51
What are the steps in systematic coding?
1. Transcription of verbal data
2. Immerse yourself in the data (i.e. read the transcripts several times)
3. Chunking into single meaningful units
4. Close reading and detailed coding
5. Analysis and sorting of similar codes into categories
6. Recode text
7. Finalize coding system = i.e. list of (sub)categories with definitions)
RS: Usability & User experience Evaluation
flashcards
Decks in class (5)
# Cards
