Chapter 9 - Evaluation

Question 1

Evaluation

Answer 1

• Summative evaluation
• – assesses for aspects of usability of finished system
• – good system can evolve through generating, representing, testing, refining ideas
• Formative evaluation
• – usable interfaces never produced spontaneously by designer
• – process produces good usability through evolution
• – data derived will point to strengths & weaknesses of proposed system
• – provides ways of learning about design options, user group, task space by deep analysis of proposed systems

Question 2

Observational evaluation

Answer 2

Choice between user observation & user experimentation

• observation
• • Emphasis on real tasks and environment
• • real users and system tested and observed
• experimentation
• • rigour & replicability of results

Question 3

Observational evaluation problems

Answer 3

• hawthorne effect
• observer bias
• halo effect

Question 4

Designing Observations

Answer 4

• tasks
• users
• environment
• 1 or 2 significant functional requirement

Question 5

Verbal Protocol

Answer 5

record that contains

• description of environment
• tasks completed by user
• users background & demographic details
• users comments, body language, facial expressions record
• aspects of interface that responses relate to

Question 6

How To Conduct Session

Answer 6

• choose task
• select users
• explain purpose to users
• conduct evaluation

Question 7

Transcript

Answer 7

• data available at the end of session
• commentary, including user’s interpretation what’s seen, expectations and response to what happened
• – categories include frequency, fundamentality

Question 8

stock walkthrough questions to be asked at each step of task

Answer 8

• will user be trying to achieve right effect
• will user know correct option is available
• will user realise correct option
• will user understand feedback to correct option

Question 9

summative vs formative evaluation

Answer 9

summative
-assesses finished system for aspects of usability
formative
-produces good usability through evolution, deep analysis of proposed designs

Question 10

comments categories (2)

Answer 10

• frequency

- fundamentality