Explain ‘SUS’.
System Usability Scale.
Measures subjective usability using a 5-point Likert scale.
What are the 10 SUS questions?
- I think that I would like to use this system frequently.
- I found the system unnecessarily complex.
- I thought the system was easy to use.
- I think that I would need the support of a technical person to be able to use this system.
- I found the various functions in this system were well integrated.
- I thought there was too much inconsistency in this system.
- I would imagine that most people would learn to use this system very quickly.
- I found the system very cumbersome to use.
- I felt very confident using the system.
- I needed to learn a lot of things before I could get going with this
system.
How is SUS evaluated?
Likert scale translates into numerical values, and these values are averaged.
Why is SUS controversial?
- Going from agree to strongly agree is different from neutral to agree - overall score does not change
- Some people may tend to answer moderately (agree, disagree), and some may tend to answer strongly.
What are the 3 measures of ‘central tendency’?
- median
- mode
- arithmetic Mean
Which variables is ‘median’ suitable for?
Ordinal variables, with an ordering
In what cases is ‘median’ more suited than ‘arithmetic mean’?
Skewed distributions, data with outliers
What needs to be done for representative scores?
Random sampling, as different users will give different scores.
Why is ‘confidence interval’ used?
To convey the quality of the obtained results.
What does ‘confidence interval’ indicate?
The range in which a parameter (e.g., median) is expected to lie.
What does ‘confidence level’ (e.g. 95%) represent?
Probability that the parameter is within the interval
“Confidence intervals are always between original data points.”
TRUE or FALSE?
TRUE
What is ‘significance testing’ used for?
Helps determine whether differences between measures are meaningful.
Explain ‘TWO-sided sign test’.
- Tests if one measure is better than another for most of the population.
- Computes confidence interval for the median of the difference.
- If confidence interval excludes zero, result is significant.
Explain ‘ONE-sided sign test’.
- Useful when a specific measure is expected to be better.
- Requires fewer measurements for significance.
- Interval is open on one side, making significance easier to achieve.
- Result is significant if p-value < 0.05
Explain ‘Wilcoxon Signed-Rank Test’.
- Offers higher statistical power than the sign test.
- Takes differences into account and is suitable for cases with less clear results.
- Provides a w-value based on ordered differences and signs.
- The critical w-value determines significance, with lower w-values indicating significance.
What is a ‘Type I error’?
False Positive
Null hypothesis is rejected when it should not be rejected.
What is a ‘Type II error’?
False Negative
Null hypothesis is not rejected when it should be rejected.
How is ‘w-value’ for ‘Wilcoxon Signed-Rank Test’ calculated?
- Order differences according to their absolute size:
-0.3, 0.6, -2.1, 2.2, 3.4, 5.1, 5.7, 6.1, 6.8 - Get an ordered list of signs
- + - + + + + + + - 1 3
w-value = 1 + 3 = 4 - Look up in table for critical w-value
For n = 9, maximally allowed w-value is 5 for p = 0.05, so result is significant.
