Lecture 5

Question 1

What does usability try to achieve? and what are metric examples?

Answer 1

Usability:

1) Finds usability problems
2) Takes quantitative measure of usability and user experience (metrics)
3) Answers research questions about the usability and UX of systems

Metrics:

• Metrics are quantitative measure of usability.
• They measure some aspect of the user’s interaction with the system or perceptions of it.
• Effectiveness (proportion of tasks successfully completed)
• Efficiency (time to complete each task)
• Learnability (time required to obtain error free performance)
• Memorability (proportion of tasks successfully completed a certain time after learning to use the system)
• Satisfaction, User Experience (perceptions of the system, measured on rating scales)
• Number of usability problems

Question 2

What is formative and summative evaluation? (4 points)

Answer 2

Formative evaluation

• Results of evaluation used to improve product
• Performed throughout the interaction design process
• Results of evaluation are fed back into the design
• Concerned with product improvement
• Formative evaluations focus on identifying usability problem; often qualitative data

Summative evaluations

• Usually performed at the end of the design process
• Concerned with metrics, taking measurements
• Results used to make judgements about the design, to compare against other metrics, for benchmarking and quality assurance
• Summative evaluations focus on measurements; often quantitative data

Question 3

What are 4 ways to assess usability with users?

Answer 3

Usability testing - in lab or “in the wild” or online

Query-based evaluations - questionnaires, interviews

Physiological measures

Experiments and A/B testing

Question 4

What are 2 ways to assess usability without users?

Answer 4

Heuristic Reviews

Benchmark reviews

Question 5

How would you PLAN (6 points) and RUN a test session? (8 points)

Answer 5

• Goals for the usability test (What are you seeking to discover? Usability problems? What findings will you deliver? Using what metrics?)
• The product/prototype to evaluate
• Task scenarios - what you will ask participants to do
• Participants
• Information sheet and consent form for participants
• Questionnaires, interviews

1. Welcome and brief participants
2. Gain participants’ informed consent
3. Possibly administer pre-test questionnaire for demographic info
4. Give introductory info and training
5. Ask participants to complete tasks, one at a time, often with a think aloud
6. Observe and record this actual use of system, remaining neutral
7. Possibility conduct de-briefing interview - follow up on difficulties
8. Administer post-test questionnaire, including experience ratings

DATA CAPTURE

• Participant’s interactions with system (computer logging, screen capture or video)
• Participants reactions
• Thinkaloud or other comments
• Timings etc
• Make notes