Lecture 8

Question 1

Give 6 user testing metrics

Answer 1

• Effectiveness (proportion of tasks successfully completed)
• Efficiency (time to complete each task)
• Learnability (time required to obtain error free performance)
• Memorability (proportion of tasks successfully completed a certain time after learning to use the system)
• Satisfaction, User Experience (perceptions of the system, measured on rating scales)
• Number of usability problems

Question 2

At what stage of the design process are questionnaires more appropriate?

Answer 2

During the evaluation stage

Question 3

Give some examples of the different types of questionnaires

Answer 3

System Usability Scale

Standardised Universal Percentile Rank-Questionnaire

User Experience Questionnaire: UEQ

UMUX-Lite

Question 4

What are the two kinds of question scales in questionnaires?

Answer 4

Likert Scale
- Users rate their agreement with a statement that expresses a view such as “This product is beautiful”

Semantic Differential Scales
- Users score a neutral statement

Question 5

What is an advantage and a disadvantage of a questionnaire?

Answer 5

Advantage:
Get user’s views directly; get at the UX; simple and cheap; potential to address many users, including remote users; gain quantitative data easily.

Disadvantage
May be a rationalised account rather than a wholly accurate one; scope limited to what you ask about

Question 6

What are Nielsen’s 10 Heuristics?

Answer 6

1. Visibility of system status
2. Match between system and real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognise, diagnose and recover from errors
10. Help and documentation

Question 7

What are some pros (2) and cons (5) for Heuristic Evaluations?

Answer 7

Pros:

• Easy to plan, rapid to perform, low cost
• Tend to identify major usability problems

Cons:

• Experts lack users’ tasks and contextual knowledge
• Experts may miss novices’ problems - different knowledge and mental models
• Results depend on expert’s knowledge of HCI, product and domain
• Better results if use more than one expert; 3-5 sometimes recommended
• But experts can be expensive and difficult to find

Question 8

What are the 9 steps of an evaluation plan?

Answer 8

1) What the goals are
2) Which evaluation methods will be used
3) What kind of data will be collected
4) How the data will be analysed
5) Who the participants (users, experts) will be
6) What they will do
7) What materials are required
8) What equipment is needed
9) Where the evaluation will take place