Module 7 - Validation

Question 1

Upstream vs Downstream Validation

Answer 1

Downstream = during design (going from outer to inner levels)
Upstream = final visualisation implemented, validate design (going from inner to outer)

Question 2

Algorithm Validation

Answer 2

Downstream: Analyze computational complexity
Upstream: Experimental Study

Question 3

Visual Encoding Validation

Answer 3

Downstream: Justification of choices (Directly derives from task abstraction)

Upstream: Informal Usability Study and Lab Study

• Informal Usability Study
  using qualitative (user experience, opinion) or quantitative data (measurable performance indicators

-> ICET method
- assessing value to a visualisation (insight,time,essence,confidence)
1. Enroll 5 participants
* Visualization users, not necessarily domain experts
* Must be familiar with the data used in the evaluation
* Arrangement with individual sessions
2. Set up the visualization(s) with some dataset(s)
* This method can also be used to compare
different visualizations
* “Teach” each participant the visualization
3. Participants rate each visualisation based on heuristics

？Few users
？Responses might be biased towards a positive outcome
？Subjective interpretation of the heuristics

• Lab Study
  Quantitative: Performance/Accuracy of users
• Laboratory experiment in controlled conditions
• Allow strong conclusions
• Requires careful formulation of hypotheses
  Qualitative: User experience and/or opinions

Question 4

Data/Task Abstraction Validation

Answer 4

Downstream = Wrong data/task abstraction
Upstream = Field Study and Case Study

• Field Study
  deploy tool into real world setting and ask experts to integrate in their workflow, ask if they can solve tasks using our tool and if workflow changed in any way
• Case Study
  finding out whether can make meaningful insights with our visualisation, test on target users and collect evidence

Question 5

Domain Validation

Answer 5

Downstream = Observe target users
Upstream = Measure adoption