Lecture 13: User Testing

Question 1

What are the THREE kinds of user tests?

Answer 1

1. Formative Evaluation
2. Field Study
3. Controlled Experiment

Question 2

What is the formative evaluation method?

Answer 2

The formative evaluation method finds problems for the next iteration of the design, it involves evaluating the prototype or implementation in lab on chosen tasks. The results will produce qualitative observations.
There are 6 stages to conducting the formative evaluation method of user testing.

Question 3

What is the Field Study method?

Answer 3

The field study finds problems in context, and evaluates the working implementation in real context on real tasks. The field study results will provide mostly qualitative observations with some quantitative observations.

Question 4

What is the Controlled Experiment method?

Answer 4

The controlled experiment tests a hypothesis, and evaluates working implementation in controlled lab environment on chosen tasks. The controlled experiment results in mostly quantitative observations: time, error rate, satisfaction etc.

Question 5

What can usability be measured in?

Answer 5

1. Qualitative - stress responses, subjective satisfaction, perceived effort or difficulty.
2. Quantitative - time on task, success / failure rates and effort: number of clicks / perception of progress.

Question 6

What is the user observation method?

Answer 6

1. Tester selects the task to be completed
2. User is given the task and access to the interface
3. User navigates through the system and tries to complete the task.
4. User speaks all thoughts aloud.
5. Tester watches the user and records all spoken thoughts and reactions for later evaluation.

Question 7

What is the Think Aloud method?

Answer 7

The think aloud method is when the user testing is taking place the user speaks their thoughts aloud, the tester stays quiet and does not respond to questions. If the user is quiet, the tester can prompt them.

Question 8

What are the FOUR advantages of the Think Aloud method?

Answer 8

1. Gives insight into what the user is thinking
2. Usable in early iterations
3. Shows whether user understands system image
4. Most widely used evaluation method in industry

Question 9

What are the THREE disadvantages of the Think Aloud method?

Answer 9

1. May alter the way users do the task - take longer and slower to complete tasks
2. Unnatural - awkward and uncomfortable
3. Hard to talk if they are concentrating.

Question 10

In the formative evaluation method, what are the six stages to conducting the user test?

Answer 10

1. Develop the test plan.
2. Select and acquire participants
3. Prepare test materials
4. Run a pilot test
5. Conduct the real test
6. Analysis and final report

Question 11

What is the facilitator’s role?

Answer 11

The facilitator does the briefing, provides the tasks, prompts the user to think aloud by asking questions and controls the session and prevents interruptions.

Question 12

What is the user’s role?

Answer 12

The user needs to complete the task and should think aloud.

Question 13

What is the observer’s role?

Answer 13

The observer is quiet and take notes.