Experimental Design

Question 1

what is ASLICE

Answer 1

• social ACCEPTABILITY
• walkup use at the START
• LEARNING curve
• INTRUSIVENESS
• CONTEXT of use
• ERROR prevention and robustness

these are issues to consider when developing a new deisgn technique

Question 2

why is a good baseline essential?

Answer 2

needs to be a valid study. Comparing your new cool thing with an old thing that no one uses isn’t a good baseline

Question 3

what is internal validity?

Answer 3

the study must have reliable experimental outcomes (repeatable)

Question 4

what is external validity?

Answer 4

an experiment is external valid if it has actual use contexts in the real world

Question 5

what is the purpose of an experiment?

Answer 5

to find measurable differences in the dependant variables between levels of the IVs

Question 6

what are prominent threats to experimental validity?

Answer 6

• internal validtiy
• external validity
• reproducability
• study heterogeneity
• – can the study be compared against previous and future ones?

Question 7

what is the standard method description? why is it important?

Answer 7

important because the study must be reproducible

setup
- an explanation of the design of the experiment
participants
- how they were sampled and their traits
apparatus
- hardware and software used
material
- text or other material used
procedure
- how experiment was conducted. what did the participants do and in what order?

Question 8

What is between-subjects design? pros and cons

Answer 8

each participant is exposed to only one condition

pros
no risk of skill transfer therefore no need to counterbalance or check for asymmetrical skill transfer

cons

• varience is not controlled (e.g. participant 1 might be an expert but 2 isn’t)
• therefore demands more participants than within-subjects for an observed difference to be statistically significant
• so gets expensive

Question 9

when might you want to use a between-subjects design?

Answer 9

if there is any risk of asymmetrical skill transfer

Question 10

What is within-subjects design? pros and cons

Answer 10

each participant is exposed to all conditions

pros

• variance is controlled within the participant
• requires fewer participants

cons

• more care is required to get design right; counterbalancing of start conditions
• risk of asymmetical skill transfer

Question 11

how do you offset skill transfer?

Answer 11

split the participants so their starting conditions vary e.g. have 50% of them try option A first and 50% try B first
latin squares can be used when combinational explosion is an issue

Question 12

what is a mixed design?

Answer 12

a combination of between and within subjects design

• used when there are multiple conditions that require testing

Question 13

how is the variance calculated?

Answer 13

o2 = 1/N x the sum of (xi - xbar)2

Question 14

what is asymmetircal skill transfer? what do if occur?

Answer 14

occurs when presenting condition A before B helps the performance of B more than presenting B before A helps A

if it occurs a within-subjects design can be transformed into a between subjects design by discarding half the data