What does significance mean?
- statistical significance
- although scores may look different, they might not be statistically different
What is the significance’s impact?
- practical significance
- practical decisions are more subjective and based on the reasons for conducting the study in the first place
Practical significance can be described as ____ ____ in research.
effect sizes
Internal validity:
- can you conclude that the findings are based on the manipulation of the independent variable?
- is the relationship causal?
External validity:
is there an interest in generalizing the findings to other populations or settings?
Construct validity:
can we generalize to the constructs?
Logical validity:
have the researchers provided a convincing justification for all aspects of the research project?
Logical validity refers to:
- the quality of the researchers’ arguments
- their application of theory to support the needs of for the study
- the appropriate interpretation of results based on the data
Logical validity relates to:
- purpose statement
- research question/hypotheses
- methodological coherence
To assess logical validity, we must examine the ____ ____ of researchers arguments and ____ ____ from the beginning to the end of their research.
- logical flow
- decision making
In assessing logical validity, we need to read and critically evaluate the entire research report, which includes:
- intro and lit review
- study purpose
- hypotheses
- research methods (study design, participants, measures, data analysis)
- results
- conclusions
_____ and _____ error play a role in logical validity.
- type I
- type II
Another way to support logical validity is to ensure ____ research is being ____ appropriately.
- previous
- cited
6 steps to ensuring previous research is being cited appropriately:
- read and critically evaluate all literature to be cited
- clearly note the kind of evidence claimed for each citation
- clearly summarize experimental effects
- critically present all sides and consensus in research
- emphasize reliable sources and original evidence
- strive for perfection in referencing details
_____ is at the heart of quantitative research.
measurement
Without strong measures of variables, the resulting research findings are…
of little use
We want measures that are both ____ and ____.
- reliable
- valid
5 potential ways to evaluate construct validity:
- have the tool reviewed by experts in the field
- examine consistency in the scores across multiple measurements
- compare groups with known differences to detect differences on the tool
- correlate the tool with other measures that already have well-established construct validity evidence
- test how sensitive the tool is
