Reliability
Consistent measurement
Validity
Does it measure what it should
Internal reliability
Consistency of the measured items
- perfect internal reliability: any set of items from scale could be selected and will provide a measure that is more or less the same as any other group of items
- internal consistency: respondents answer related items similarly
- in some cases measurement that covers several different aspects is desirable (creates low internal consistency)
Types of internal realiability
- traditional approach: Pearson correlation (each half is similar)
- split-half reliability: each participants summed scores are calculated separately for both halves of items
- odd- even realiability: one half of items is odd numbered the other even
- spearman-brown formula: estimation of true split half with correlation to length of scale
Reply on particular items in two halves
Cronbach Alpha
Average of every possible half correlated to every other possible half
A>0.70 (adequate)
A>0.95 (too high)
Realiability coefficients
- removing items causing internal reliability to be lower
- common method: computer cronbach a for total scale and see how it changes when removing items
- stop if satisfactory value is achieved
Item total correlations
- correlations of participants score on each item with their overall score
- if all of the individual items on the scale are measuring the same structure, all items correlate with total score on scale
- removing items with low item total correlations
- low item correlation: inconsistent item with possibility measures a diff psych construct
Item discrimination between extreme groups
- removing those items which have low discrimination with people scoring low or high
- high reliability: if items are differently answered by highscorers -> item retained
- low reliability: items which highscorers are the same as lowscorers -> item dropped
Reliability over time
- consistency of measure at different time across different circumstances
- test-retest reliability:
- > repeated measurement/ longitudinal design
- > longer interval between: more opportunity for within subject
- > carry over effect: test and immediate retest
Face validity
You read it and you check if the item measures what you want to measure
High face validity: makes people more motivated but also more prone to lying
Content validity
Optimal content validity: careful collections of items that is diverse for the concept being assessed
-> alternative: content validity achieved by experts decision
Criterion validity
- exploring wether scores on a given psychological test are successfully related to know criteria
-> concurrent
-> predictive
Concurrent validity
Assessed against a criterion measuring same construct
- calculating correlation of the test with another test for same construct
- > ideally a strong correlation (but if it’s a new theoretical construct, a lower correlation is better)
Predictive validity
- ability of measure to predict future events
Not all measurements intend on predicting future
Construct validity
Aim: specify the nature of psychological construct that underlies given measurement
Support for construct validity: depends on assumptions regarding nature of construct
Triangulation
Two different measures with same construct will correlate with variable the construct might relate to
-> combination of concurrent and predictive validity
(If A and B correlate-> concurrent)
(If A and B correlate to X -> predictive)
Known groups validity
- evidence of optimal validity: can be established that scores on a measure differ in predictable ways between 2 specified groups
Convergent validity
-measures of given psych construct should relate irrespective of the nature or more of the measure
Discrimination validity
- if measures measure different construct no correlation is expected
