Week 4

Question 1

Validity

Answer 1

the degree to which a test or instrument measures what it purports

Question 2

Validity

Logical / Face Validity

Answer 2

do the methods and approach make sense?

Question 3

Validity

Content Validity

Answer 3

Does the test fully represent the domain of the concept it is intended to measure?

Question 4

Validity

Criterion Validity - Concurrent

Answer 4

correlating an instrument with a criterion administered at the same time

Question 5

Validity

Criterion Validity - Predictive

Answer 5

correlating an instrument with a criterion administered in the future

Question 6

Criterion Validity - Predictive

Cross- Validation

Answer 6

• generate a predictive equation with 1/2 of the sample
• confirm the prediction equation works with other 1/2 of sample

Question 7

Validity

Construct Validity

Answer 7

degree to which the scores from a test measure the hypothesis construct

Question 8

Measurement Reliability

Answer 8

• refers to the consistency and stability of a measurement tool or method
• indicates how dependably a method measures something over time, across different observers, or in various contexts

Question 9

What are common sources of measurement errors?

Answer 9

• testing
• instrumentation
• scoring
• participant

Question 10

Reliability Tests

Test-retest Reliability

Answer 10

measures the consistency of results when the same test is administered to the same sample at different points in time

Question 11

Reliability Tests

Interrater Reliability

Answer 11

assesses the degree to which different observers or rates agree in their assessments

Question 12

Reliability Tests

Parallel Forms

Answer 12

comparing two different versions of a test that are designed to be equivalent

Question 13

Reliability Tests

Internal Consistency

Answer 13

examine the consistency of results across items within a test
* a common measure of internal consistency is Cronbach’s alpha, which assesses how well the items on a test measure the same construct

Question 14

Common Statistical Approaches

Test-retest Reliability

Answer 14

• Pearson correlation coefficient
• Intraclass correlation coefficient (ICC)
• Coefficient of Variation

Question 15

Common Statistical Approaches

Interrater Reliability

Answer 15

• Percent agreement
• Cohen’s alpha
• Intraclass correlation coefficient (ICC)

Question 16

Common Statistical Approaches

Parallel Forms Reliability

Answer 16

• Pearson Correlation coefficient
• Split-Half method
• Counterbalancing method

Question 17

Common Statistical Approaches

Internal Consistency

Answer 17

• Cronbach’s Alpha
• Split-Half reliability

Question 18

Ranking research design from weakest evidence to strongest evidence.

Answer 18

• Animal research (does not involve animals)
• Clinical textbooks, experts opinions (not primary research)
• Case reports / case studies (no or little design)
• Case-control studies (observational)
• Cohort studies (observational)
• Randomized control trials (experimental)
• Systematic review (evidence study)
• Meta-analysis (evidence study)

Question 19

Threats to Internal Validity

History

Answer 19

External or historical event that occurred during the course of the study that may be responsible for the effects instead of the intervention itself

Question 20

Threats to Internal Validity

Maturation

Answer 20

An internal and natural process that leads participants to change on the dependent measure

Question 21

Internal Threats to Validity

Testing

Answer 21

Changes in test scores occur because of repeated testing

Question 22

Threats to Internal Validity

Differential Selection

Answer 22

Any difference between the study groups before the start of the study due to either self-selection or experimenter selection

Question 23

Threats to Internal Validity

Instrumentation

Answer 23

Changes in instrument calibration or human observers

Question 24

# Threats to Internal Validity Regression to the Mean

Answer 24

The phenomenon that if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement

Question 25

# Threats to Internal Validity Selection Interactions / Expectancy

Answer 25

Both researchers or study subjects may tend to expect one group may perform better (or worse) the other

Question 26

# Threats to Internal Validity Experimental Mortality (Attrition)

Answer 26

People can drop out of the study for any reason; if reason is related to treatment status it may cause bias

Question 27

What are threats to External Validity?

Answer 27

* selection bias * pretesting * experimental setting * multiple treatment

Question 28

# Threats to External Validity Selection Bias

Answer 28

* Who is included/excluded * Race/ethnicity, age, sex, lifestyles, SES * Disease characteristics

Question 29

# Threats to External Validity Pretesting

Answer 29

If you pretest, then when participants take the "true test" their responses may differ compared to if they never took the a pretest

Question 30

# Threats to External Validity Experimental Setting

Answer 30

* Participants performance improves just b/c they know they are being watched * Hawthorne Effect

Question 31

# Threats to External Validity Multiple Treatments

Answer 31

Receiving on treatment may influence how you respond to other treatments

Question 32

Random Control Trial (RCT)

Answer 32

* A study design where participants are randomly assigned to either the intervention group or the control group * randomization minimizes bias, allowing for causal inferences

Question 33

Quasi-Experimental

Answer 33

* A study design that looks at the effect of an intervention without random assignment * while it can still compare groups, the lack of randomization makes it more prone to bias

Question 34

Natural Experiment

Answer 34

* A study that takes advantage of "naturally occurring circumstances in which subsets of the population have different levels of exposure to a supposed causal factor, in a situation resembling an actual experiment" * Provides an alternative when it is impossible or unethical to assign treatment