1
Q
The two types of Experimental Designs are…
A
- Causal
2. Correlational
2
Q
Causal Experiments mean that…
A
one variable directly or indirectly influences another
3
Q
Correlational Experiments mean that…
A
changes in one variable accompany changes in another
4
Q
What are the Three Criteria for Causal Relationships?
A
- co-variation between X and Y variables
- Temporal sequence, X occurs before Y variable
- Control of third (extraneous or confounding) variables
5
Q
When the behaviors and characteristics of the researcher influence the reactions of participants, it is called…
A
Researcher Error
6
Q
When the personal characteristics and experiences of participants influence their responses, it is called…
A
Participant Error
7
Q
When aspects of the environment influence scores, it is called….
A
Environmental Error
8
Q
When aspects of the stimuli presented (the IV) or measurement procedure employed influence scores, it is called…
A
Measurement Error
9
Q
What is the definition of Third Variables?
A
- Any variable that can potentially influence results, but is NOT intended by the researcher
2, Any variable that may compete with the IV in explaining the outcome of the study (DV)
10
Q
How do Third Variables influence research?
A
- By influencing the scores & thus the mathematical relations produced.
- Leads to errors when interpreting and generalizing the study, because the variables that we think are operating are not those that are really operating
11
Q
What is a Dis-simulation effect?
A
Third variable result of increase in scores
12
Q
What is a masking effect?
A
Third variable result of decreasing scores
13
Q
What is the definition of an Extraneous Third Variable?
A
- Any variable that can potentially influence results, but is NOT intended by the researcher
2, Any variable that may compete with the IV in explaining the outcome of the study (DV)
14
Q
How do Extraneous Third Variables influence research?
A
- By influencing the scores & thus the mathematical relations produced.
- Leads to errors when interpreting and generalizing the study, because the variables that we think are operating are not those that are really operating
15
Q
What is a masking effect?
A
Third variable result of decreasing scores
16
Q
How does an Extraneous Third Variable fluctuate?
A
- unsystematically, randomly, no consistent pattern
2. systematically, consistent pattern
17
Q
What are the two types of Third Variables?
A
- Extraneous Variables
2. Confounding Variables
18
Q
What is a Confounding Variable
A
- an extraneous variable the DOES compete with the IV in explaining the outcome of the study
- Systematic variation in an extraneous variable
19
Q
List FOUR examples of Confounding Variables
A
- Hawthorne (Attention) Effect
- John Henry Effect
- Placeo Effect
4 Demand Characteristics
20
Q
List FOUR examples of Confounding Variables
A
- Hawthorne (Attention) Effect
- John Henry Effect
- Placeo Effect
4 Demand Characteristics
21
Q
What is another name for the Attention Effect?
A
Hawthorne Effect
22
Q
What happened in the Hawthorne Effect?
A
Workers productivity went up when lighting was increased AND when it was decreased
23
Q
What is the interpretation of the Hawthorne Effect?
A
workers respond to researcher attention whether or not the lighting was increased or decreased
24
Q
What controls were used in Hawthorne Confound?
A
- Experimental group
- control receiving attention
- control NOT receiving attention
25
What is the John Henry Effect?
possibility that the control group might become aware of its 'inferior' status and respond by trying to outperform the experimental group
26
What is the John Henry Effect controlled by?
concealing the control group's knowledge of its status
27
What is the Placebo Effect?
tendency of individuals to improve (or feel improved) simply because they know they are being treated
28
Placebo Effect controlled by...
giving the control group a placebo which provides treatment for the control group
29
What is a Demand Characteristic?
a cue that lets participants know the expected outcomes of an experiment
30
Demand Characteristics are controlled by....
concealing researcher expectations from participants
31
Confounding is a threat to _________ ____________?
Internal Validity
32
What is Internal Validity?
1. the degree to which the relationship between the scores reflects only the relationship between the intended variables
2. Changes in the Y (DV) related directly and only to changes in the X (IV)
33
What is Internal Validity?
1. the degree to which the relationship between the scores reflects only the relationship between the intended variables
2. Changes in the Y (DV) related directly and only to changes in the X (IV)
34
List 7 Common threats to Internal Validity.
1. History
2. Maturation
3. Instrumentation
4. Testing
5. Statistical Regression
6. Selection Bias
7. Mortality
35
What are 2 factors in Historical threats to Internal Validity?
1. environmental influences between pre and post tests
| 2. non-treatment events occurring between observations
36
Name 2 Factors in Maturational Threats to Internal Validity.
1. participants age between pre and post tests
| 2. performance changes due to aging and confounds the effect of the treatment
37
Name 2 Factors in Instrumentational threats to Internval Validity.
1. Potential changes in measurement procedures from pretest to post test administration
2. unobserved change in observer criteria or instrument calibration confound the effects of treatment
38
Name 2 Factors in Testing Threats to Internal Validity
1. learning occurs during pretest and affects post test results
2. testing prior to treatment changes how people respond in post-testing
39
What is the statistical regression threat to internal validity?
participants selected for treatment on the basis of their extreme scores tend to move closer to the mean on retesting
40
What is the Selection Bias threat to internal validity?
nonequivalent initial groups used
41
What is the Mortality threat to internal validity?
differential loss of participants from the groups to be compared resulting in nonequivalent groups
42
What is the single best approach to controlling threats to Interval Validity?
Experimental Design that incorporates random assignment of participants to experimental and control groups
43
List FOUR Common threats to External Validity.
1. Interaction of Selection Bias with IV
2. Reactive effects of experimental arrangements
3. Reactive effects of testing (pretest sensitization)
4. Multiple Treatment Interference
44
Describe the Interaction of Selection Bias with IV (threat to External Validity)
1. If a biased sample of participants is used in an experiment, a researcher will not know whether the effects of the treatment can be expected if the treatment is administered to the entire population
2. Results of the study may only pertain to the particular type of participants that were included in the study
45
Interaction of Selection Bias with IV is controlled by.....
random selection
46
Describe the Reactive effects of experimental arrangements (Threat to external validity)
1. If the experimental setting is different from the natural setting in which the population usually operates, the effects that are observed in the experimental setting may not be generalized to the natural setting
2. The presence of observers, instrumentation, or the lab environment that affects the participant but would not occur if the participant had been in a natural setting
47
Reactive effects of experimental arrangements is controlled by....
experiments conducted under natural conditions when possible
48
Describe Reactive effects of Testing (pretest sensitization). (Threat to External Validity)
1. Possibility that the pretest might influence HOW the participants respond to the experimental treatment
2. Pretesting may decrease or increase the sensitivity of the participants to the IV. Thus, one might not be able to generalize to non-pretested individuals.
49
Reactive effects of testing is controlled by....
conducting a true experiment without pretests
50
Describe Multiple Treatment inference in Threats to external Validity.
1. Occurs when a group of participants is given more than one treatment
2. Treatments given earlier in an experiment might affect the effectiveness of treatments given later to the same participants
3. Cannot generalize to single treatment conditions
51
Multiple treatment interference is controlled by....
not using multiple treatments in the experiment
52
Question answered by the meaning of Internal Validity is...
Is the treatment responsible for the observed change?
53
Question answered by the meaning of External Validity is...
To whom can the results be generalized?
54
What happens to internal and external validity when a control is used to reduce the effect of confounding and extraneous variables?
1. Internal Validity increases
| 2. External Validity decreases
55
What happens to internal and external validity when steps are taken to make the experimental setting as natural as possible?
1. Internal Validity decreases
| 2. External Validity increases
56
Who are Campbell and Stanley?
developers of symbols used to represent research designs
57
What is the symbol for Random assignment to groups?
R
58
What is the symbol for Observation or measurement?
O
59
What is the symbol for Experimental treatment?
X
60
What is the symbol indicating a control condition?
(blank space)
61
Define a control condition.
1. May contain some kind of treatment.
| 2. is the standard by which the effectiveness of the experimental treatment is judged
62
What does the symbol X stand for?
Experimental treatment
63
What does the symbol O stand for?
Observation or measurement
64
What does the symbol R stand for?
Random assignment to groups
65
______________ ___________ designs are characterized by poor internal validity.
Pre-experimental
66
Pre-experimental designs are characterized by _________ ________ __________.
Poor internal validity
67
What are Pre-experimental designs used for?
preliminary pilot studies in which researchers are trying out potential treatment and measuring tools to learn about acceptability and accuracy
68
List 3 examples of Pre-experimental designs.
1. One-Group Pretest-Posttest Design
2. One-Shot Case Study
3. Static-Group Comparison Design
69
Define One-Group Pretest-Posttest Design
one group is:
a. pretested
b. treated with IV
c. post-tested
70
Define One-Shot Case Study
One participant is:
a. treated with IV
b. Post-tested
71
Define One-Shot Case Study
One participant is:
a. treated with IV
b. Post-tested
72
Define Static-Group Comparison Design
```
Group A
a. Treated with IV
b. Post-tested
Group B
a. Post-tested
```
73
Quasi-experimental Designs are of intermediate value for exploring ___________ and _________ relationships.
cause and effect
74
____________ -__________ designs are of intermediate value for exploring cause and effect relationships.
quasi-experimental
75
Quasi-experimental designs are used when...
random assignment to groups is not possible
76
List 2 examples of Quasi-experimental designs.
1. nonequivalent control group design
| 2. Equivalent time-samples design
77
Define Nonequivalent Control Group Design
Group A is:
a. Pretested
b. treated with IV
c. Post-tested
Group B is:
a. Pretested
b. Post-tested
78
Strengths of Nonequivalent Control Group Design
although participants are not assigned at random to the two groups, researchers often use some form of matching to increase the internal validity of the results
79
Strength of Nonequivalent Control Group Design
although participants are not assigned at random to the two groups, researchers often use some form of matching to increase the internal validity of the results
80
Define Equivalent Time-Sample Design
ABAB Design
Group/Individual A has:
a. Multiple initial observations before treatment (baseline)
b. Multiple and alternating control and experimental treatments
81
Two Strengths of Equivalent Time-Samples Design (ABAB Design)
1. researchers know that the experimental participants and control participants (who are actually the same individuals) are identical
2. Unlikely that there are threats to internal validity, since it is unlikely that they are operating only at the times when X1 is given and not when Xo is given
82
In the Equivalent Time-Samples Design (ABAB), what is the threat to external validity?
multiple treatment interference
83
True Experimental Designs are characterized by
random assignment to treatments
84
_________ __________ designs are characterized by random assignments to treatments.
true experimental
85
List 3 Examples of True Experimental Designs
1. Pretest-Posttest Randomized Control Group Design
2. Posttest-Only Randomized Control Group Design
3. Solomon Randomized Four-Group Design
86
Define Pretest-Posttest Randomized Control Group Design
Group A is:
a. randomly assigned
b. pretested
c. treated with IV
d. posttested
Group B
a. randomly assigned
b. pretested
c. posttested
87
List of 2 Strengths of Pretest-Posttest Randomized Control Group Design
1. Random assignment
| 2. Use of pretest
88
By assigning participants at random to groups, researchers are assured that there are no ___________ _____________ in the assignment.
systematic biases
89
Comparison of posttest and pretest results allow researcher to determine ____________ _________ each group gained, not just whether they are ___________ at the end of the experiment.
a. how much
| b. different
90
Define Posttest-Only Randomized Control Group Design
Group A has:
a. random assignment
b. treated with IV
c. post tested
Group B has:
a. random assignment
b. post tested
91
Define Posttest-Only Randomized Control Group Design
Group A has:
a. random assignment
b. treated with IV
c. post tested
Group B has:
a. random assignment
b. post tested
92
List 2 Strengths of Posttest-Only Random assignment
1. Random Assignment
| 2. No Pretest sensitization
93
List 2 positive factors of random assignment
1. assures compatibility of groups as with pretest-posttest control group design
2. assures that there are no biases in assignment as with pretest-posttest randomized control group design
94
Describe Solomon Randomized Four-Group Design
Group 1 is:
a. randomized
b. pretested
c. treated with IV
d. posttested
Group 2 is:
a. randomized
b. pretested
c. posttested
Group 3 is:
a. randomized
b. treated with IV
c posttested
Group 4 is:
a. randomized
b. posttested
95
Major strength of Solomon Randomized Four-Group Design
allows for all comparisons needed
96
What factor is needed to execute a Solomon Randomized Four-Group Design?
reasonably large pool of participants to start with so that a minimum of 30 per group is in the final sample
97
Why is there a minimum of 30 participants per group needed in a Solomon Randomized Four Group Design?
to ensure a sufficient number to perform statistically significant tests
98
List 3 reasons why Experimental Designs are best.
1. Co-variation between X (IV) variable and Y (DV) variable
2. Temporal Sequence, X (IV) variable occurs before Y (DV) variable
3. Control of third (extraneous or confounding) variables
99
What is 'co-variation between X (IV) variable and Y (DV) variable'?
1. researcher manipulates the IV
| 2. Then measures the results of that manipulation in the DV
100
What is "temporal sequence, X (IV) variable occurs before Y (DV) variable"?
1. researcher presents the conditions of the IV
| 2. Then measures the results in the DV
101
What is "control of third variables (extraneous or confounding)"?
1. researcher tries to eliminate or control the threats to internal validity
2. threats that cannot be eliminated or controlled are balanced through random assignment of participants to experiment/treatment and control conditions/groups
102
List 3 reasons why TRUE experimental designs are better than Quasi-experimental designs.
1. QE has NOT random assignment to groups or conditions. TE does
2. in QE the IV is not manipulated by researcher and in TE it is
3. in QE there is only some control of third variables and in TE there is
103
What is the definition of Meta-Analysis?
1. statistical methods for combining the results of previous studies on a given topic
2. mathematically synthesizing the statistical results of previous studies on a given topic
104
What does 'meta' mean?
occurring later and/or being later and more highly organized
105
What does 'analysis' mean?
synthesis
106
List 4 components in interpreting the differences found in the results of previous studies.
1. random sampling errors created by assigning participants at random to groups
2. random errors of measurement
3. systematic errors known to one or more of the researchers
4. systematic errors of which one or more of the researchers are unaware
107
List 3 important characteristics of meta-analysis.
1. statistical results are based on the combined sample size of all the studies (BIG samples sizes yield more reliable results)
2. synthesis is of the result of studies conducted by independent researchers
3. process of averaging when conducting a meta-analysis tends to decrease the effects of both the random and systematic errors identified in previous slides
108
List 2 reasons why the synthesis of meta-analysis studies is better.
1. systematic errors of one researcher are unlikely to be committed by other researchers
2. the effects of one researchers systematic errors are moderated when averaged with the results obtained by other independent researchers who have not made the same error
109
What symbols are used to denote measure of effect size?
d
| r (squared)
110
List the 3 steps of calculating effect size.
1. calculate d (r squared)
2. average the values of d (r squared)
3. if the studies involve varying number of participants, the averaged value of d (r squared) can be weighted to give more weight in the final average to those studies with more participants
111
List 3 Strengths of Meta-Analysis.
1. produces results based on large combined samples
2. synthesizes studies by independent researchers
3. objectivity of conclusions
112
List 2 potential weaknesses of meta-analysis.
1. risk of poor selection of studies
| 2. risk of publication bias
113
Because it is not possible to anticipate all potential for harm researchers must _________.
have a research committee review research plans for potential harm
114
List 2 considerations in right to privacy in research.
1. conduct research with concern for the dignity and welfare of participants.
2. psychologist inform research participants of their anticipated sharing of further use of personally identifiable research data and of the possibility of unanticipated future uses
115
List 3 measures to keep collected data confidential.
1. disclose only group level data
2. disclose in statistical form
3 change names
116
If participants are entitled to know the purpose of the research before agreeing to participate, what is the risk?
knowledge of the purpose of the study may bias the results
117
List 5 specifics to be included in the informed consent
1. general purpose of the research
2. what will be done to them during the research
3. what the potential benefit(s) to them and others might be
4. what the potential for harm to them might be
5. the fact that they may withdraw at any time without penalty, even midstream
118
List 5 specifics included in a research participant debriefing
1. Review of purpose(s) of the study and the procedures used
2. offer to share the results with the participants when the results become available
3. Reassurance that the data will remain confidential
4. Opportunity for participants to ask for information about any aspect of the study in which they participated
5. Information about how to contact the researcher in the future, either for more information or to request assistance with any harmful effects that might reveal themselves at a later data
119
What is the one caution to be aware of during debriefing?
researchers should try to identify participants who may need more help in overcoming unanticipated harm to them than a standard debriefing session provides
120
List 3 actions to be done for a participant after research.
1. provide them with information about how to contact the researcher in the future
2. offer assistance
3. request an additional follow-up visit
