Systematic error vs random error
Systematic errors are repeated in the same way throughout an investigation (using a balance incorrectly in the same way for each measurement) this can be corrected - precision describes how repeatable they are
Random error: cannot easily be corrected as it affects measurements differently
Example of nominal scale of measurement
Gender
Days of the week
Example of ordinal level of measurement
Rankings
Rating scales
Example of interval levels of measurement
Rating scales
Temperature
Example of ratio levels of measurement
Timing
Quantities: height,weight,age,length
How are constraints considered measurement concerns
Amount of time, money , available participants , equipment multiple ways to measure any construct
How do we know we are measuring the concept? How valid (VALIDITY)
Is there a degree of fit between constrict and indicator?
RELIABILITY
Are our measurements consistent and dependable?
Will respondents answer in the same way if asked again?
What are the branches of validity? (hint: 4)
give a brief example of each
Face validity: extent to which a tool APPEARS to measure what it’s suppose to
Content validity: extent to which items are relevant to the content being measured
Criterion validity/ predictive validity: extent to which responses on a measure can predict future behaviour
Construct validity: extent to which a tool measures an underlying construct
What is face validity
- when a tool subjectively appears to measure a construct
- not a good way to measure validity
- it involves peoples opinions and opinions can be wrong
“On the face of it”
Subjective assessment ( by researcher, experts)
Weak subjective method but a first step
Example: measuring interviewer skill
Maintain eye contact
Use neutral probes
What is content validity and an example ?
- extent to which the individual items on a test are relevant to the content area it is testing
- Does the measure cover the entire range of meanings included in the concept?
- Based on judgement
Example: you wouldn’t put a spelling question on a math test
What is criterion validity?
what are the three types of criterion validity?
Checking against an external criterion believed to be another indicator of same construct
Predictive validity
Concurrent validity
Known groups validity
What is predictive validity
give an example
- a type of criterion validity
- when a tool can predict certain behaviors
- does the measure predict some future criterion that it’s expected to predict?
Example: does attendance at biol1900 lectures accurately predict student performance on exams?
Concurrent validity
- a type of criterion validity
- does the measure relate to some known criterion concurrently?
example: Do scores on a measure of health-related quality of life correspond to the ratings based on clinician interviews?
Known groups validity (hint: type of … validity)
(hint: differentiate)
give an example
- type of criterion validity
- does the measure differentiate people in the way you would expect?
example: Does grip strength differentiate between those of low and high risk of cardiovascular mortality?
Construct validity (hint: how does it relate to other constructs?) what are two types of construct validity?
- extent to which a tool measures a construct
- hard to prove
- Relates to other constructs in a way that is expected based on theoretical relationships
Convergent validity
Divergent validity
Convergent validity (hint: type of … validity)
What is convergent validity associated with?
Give an example
- type of construct validity
- Associated with other measures that it should be related to
example: Do scores on a vertical jump test correspond to wall sit test times (leg strength)
Divergent validity (hint: type of ... validity) give an example
- type of construct validity
- Does not associate with measures of other constructs as closely as it does with other measures of same construct
example: Do BSS scores relate more to measures of sit and reach than they do to standing long jump?
What are the three main points (degrees) of reliability
Consistency
Repeatability
Agreement
What is consistency (hint: one of the three main degrees of …)
- a degree of reliability
- Degree of consistency in a measurement
- Do all items on the measure reflect the same underlying construct?
- Internal consistency reliability (cronbachs alpha)
What is the degree of repeatability of measurement (hint: one of the three main degrees of …)
give an example
- reliability
- Does the same measurement technique give the same result each time you use it
- Test-retest reliability (Pearsons correlation, r)
Problems with reliability (hint: 3 problems)
- Internal consistency (difficult items, unrelated items) -Test retest (memory effects, practice effects, time interval)
- Inter rater: (non standardised or no instructions, different experiences, need calibration)
Type 1 error
- Find a significant relationship but one does not exist in the real world
- “false positive”: the error of rejecting a null
hypothesis when it is actually true.
Type 2 error
- You find no significant relationship when one does exist in the real world
- “false negative”: the error of not rejecting a null
hypothesis when the alternative hypothesis is the true state of nature
