Factorial Design

Question 1

Describe a Factorial design:

Answer 1

• More than 1 independent variable, i.e., more
  than 1 factor (grouping variables)

Question 2

Describe this Factor Design:

Answer 2

Two Factor Design:
2 factor = 2 independent variables,
regardless of number of levels in each factor
* Described as number of levels x number of levels
2x2
2x3
3x3
* Analysed using 2-way (factor) ANOVA

Question 3

Describe this Factor Design:

Answer 3

Three Factor Design
3 factor design would have 3 independent
variables
* 2x2x2, 3x2x2, etc.
* Analysed using 3-way (factor) ANOVA)

Question 4

Independent variables are all_____________________

Answer 4

– IV are all independent

E.g., Do classical and rock singers show different rates of vocal polyps and does this vary with training or no voice training.
4 gpr: classical with, classical without, rock with, rock without

Question 5

Describe the IV of within repeated measure:

Answer 5

• IV is dependent

E.g., Which hearing aid is preferred and does this make a difference with a trial period
Each participant tested on three h.a. immediately and after a month’s trial – both within (aid type - 3 levels and time - two levels)

Question 6

Give an example of a Mixed Design

Answer 6

– Both between and within

E.g., Does the ‘Marvelous Metaphors’ program improve children’s comprehension of metaphors
1 grp gets program, 1 grp does not (between factor)
Test pretreatment, post-treatment, maintenance (within factor, 3 levels)

Question 7

What are marginal means?

Answer 7

• Means for each independent variable (averaged across cell)
• 4 in a 2x2 factorial design
• Main effect
  Difference between levels of 1 independent variable

Question 8

What are Cell Means?

Answer 8

• Means for each cell
• Interaction effect: combinations of independent variables

Question 9

What are the steps in 2-way (factor) ANOVA (between subjects)?

Answer 9

1. Different participants in each cell
2. Each measured once for DV
3. Statistic
   3 F-ratios
   Main effect of IV1
   Main effect of IV2
   Interaction
4. Determine if p value less than alpha
5. Post-hoc testing if more than 2 levels for any IV-
6. Post-hoc testing if a significant interaction
7. Effect sizes reported for significant differences

Question 10

How would you set up this experiment:

Do the program stream and home province make a difference in satisfaction among SCSD students?

Answer 10

1. Use a continuous scale to measure satisfaction and measure distance in mm-equal intervals.
   Unsatisfied _____________ Satisfied
2. This is a 2-way ANOVA (between subjects)
3. Split Program in HCD (2 levels) –main effect
   SLP vs Audio – 2 means
4. Split Home (Atlantic vs ON/PQ vs Western Prov) – main effect
   Atlantic vs ON/PQ vs West – 3 means
   - Need post-hocs
5. Interaction
   SLP Maritime; Audio Maritime; SLP ON/PQ; Audio ON/PQ; SLP West; Audio West – 6 means
   Need post- hoc tests

Question 11

In this example: 2 groups with different types of dementia are treated using 2 different interventions

How many IV groups?
What is DV?
What posthoc comparisons are interesting to test?

Answer 11

4 independent grps
Gain scores are analyzed (DV) using a 2-way (factor) ANOVA
4 comparisons
4 means involved

Question 12

What is the Interaction Effect? (4)

Answer 12

• Effect of one variable not constant across different levels of the second variable
• Interaction between IVs has unique effect
• With an interaction effect, significant main effects should not be interpreted directly.
• Can still have significant main effects even if have interaction (but not likely)

Question 13

What are the differences between group vs. single-subject designs?

Answer 13

• Both important: convergence of evidence
• Both can be experimentally rigorous
• Group designs allow greater generalizability to the larger population as a group
• Single-subject designs provide detailed analysis of individual performance
  Help isolate characteristics that influence behavior
  Group averages never mirror individual behavior exactly
  No individual matches the ‘theoretically average” client
  Does allow ‘case-to-case’ generalization
  ‘client-centred’ practice clinically calls for individualized care
• Group designs test infrequently – can’t see natural variability and growth in measure
• Can co-occur in the same study

Question 14

When should we use a single-subject design?

Answer 14

• When withholding treatment is considered unethical
  But note treatment is delayed in SS; can use a delayed treatment control group
• When the random assignment is not possible
  When can’t get enough participants - useful for studying rare events
• When wanting a lot of detail on participants, or intervention modifications, settings, etc.
• When expecting behaviours to change when conditions change (e.g., treatment introduced/withdrawn)
• When don’t have the resources to do group
• Useful for clinical innovation

Question 15

How would you denote different stages of a study in Single-subject designs?

Answer 15

• Different letters denote different stages of a study
  A = baseline, no treatment
  B = first treatment
  C = second treatment, different than B
  B’ = first treatment with small variation
• Use subscripts to denote repetitions of a segment (e.g., A1, A2, etc.)
• Observed behaviour (DV) is plotted on Y-axis with time on X-axis

Question 16

Explain the baseline step:

Answer 16

• Repeated, continuous measurement of DV before begin any manipulation, such as a treatment
• Control Condition
• Assumed to represent how DV would behave without intervention
• Good Baseline
  Multiple measures (minimum needed 3-4; ideally more)
  Stable
  Limited variability
  No clear trend up or down
  No ceiling/floor effects
  Room for improvement/Opportunity for contrasting results

Question 17

What are the types of baseline? (4)

Answer 17

Stable (IDEAL)
Variable
Stable accelerating/decelerating
Variable accelerating/decelerating

Question 18

What are the length of the phases?

Answer 18

• Baseline
  Until get stability
  Minimum 3-4; more is better
• Treatment(s)
  Repeated, continuous measurement
  Until get stability ideally
  Minimum 3-4; more is better
  Influenced by expected rate of change
  Target
  Type of client
  Type of intervention
  Can be set by length of time/number of sessions OR until achieve criterion
  Frequency of measures is also influenced by expected rate of change

Question 19

Explain Phase A:

Answer 19

Designs or Case Studies - not Single subject
They are not hypothesis testing
Just a ‘baseline’
No experimental manipulation, only a DV
No experimental control
Careful and systematic descriptions of 1 or a few interesting or unusual case(s)
Enhanced by attempts to quantify observations
Case series: description of a group of similar cases

Question 20

Explain Phase B:

Answer 20

Observe effects of treatment over time - not SS
DV – behaviour of interest
IV – treatment
Unable to test causal relationships between the IV and the DV as no experimental control
Don’t know what DV is like without treatment, what the normal variability is
Factors other than treatment may have influenced change in DV seen during Phase B
? Threat to internal validity

Question 21

What are A-B pre-experimental designs?

Answer 21

A = baseline
B = treatment
Not experimental as no experimental control
Unable to test causal relationships between the IV and the DV as no experimental control
Here you do know what DV looks like without treatment (i.e., A)
Factors other than treatment may have influenced the change in DV seen during Phase B

Question 22

What are experimental designs?

Answer 22

• Experimental (single subject designs) are hypothesis testing

Question 23

What are the two ways to establish experimental control in a variety of ways?

Answer 23

Replications
Control Goals

Question 24

What are the benefits of increase control through replication?

Answer 24

• Increases internal validity
  The more frequently an effect can be replicated in a design, the stronger control against threats to internal validity and the more able to attribute change in DV to intervention
• Increases external validity – i.e., generalizability
  Systematically plan studies to vary across conditions
  Subjects, settings, personnel performing intervention, timing of intervention, equipment, etc.
• Interpretation: Rx is effective when
  Behaviour changes when Rx is implemented and does not change for remaining baselines
  Behaviour changes only when the Rx is implemented and does so directly or closely after implementation

Question 25

What are withdrawal designs?

Answer 25

* Replication of phases: Withdrawal designs A1 B A2 design Involves a second no treatment period (has 2nd baseline) If the DV reduces in second baseline (A2), can more easily attribute change during B to treatment

Question 25

What are withdrawal designs?

Answer 25

* Replication of phases: Withdrawal designs A1 B A2 design Involves a second no treatment period (has 2nd baseline) If the DV reduces in second baseline (A2), can more easily attribute change during B to treatment * Additional control can be built into a withdrawal design by adding more phases e.g., A1 B1 A2 B2 design Provides 2 opportunities to evaluate the effect of treatment If DV declines or levels off in A2 and then improves again in B2, demonstrates consistency of response Therefore, strong support for effect of treatment

Question 26

What would be problems with a 2nd baseline?

Answer 26

Problems with a 2nd baseline: Contrary to behaviorist theory (the underpinnings of SS designs), improvement may be irreversible Many treatments designed to develop independent learning – strategies individual can apply on their own Withdrawing a treatment that is working is potentially unethical ? Possible way around? ? Study where it would be appropriate? In Intervention Research, rather then ABA, generally viewed as ABC where C is maintenance Assumes new skill remains when intervention is withdrawn True goal of intervention

Question 27

What are multiple baseline designs?

Answer 27

Replication across subjects or conditions (settings, behaviors, therapists...) Concurrent monitoring Establish stability in baselines Introduce treatment to all at the same time Non-concurrent monitoring Arbitrarily define different baseline lengths Randomly assign length of baseline to subjects or conditions Alternative, introduce treatment to second subject after response to treatment in first becomes stabilized ? – which is preferable? Why? Controls for maturation, history, other threats to internal validity

Question 28

Explain Multiple Baseline Designs across behaviors:

Answer 28

* Two or more related yet functionaly independent behaviours (DVs) within a participant * Vary length of baseline for each behaviour Controls for external forces being responsible for change Strengthened when non-targeted behaviours remain stable until they are targeted where can you randomize?

Question 29

Explain Multiple Baselines across Behaviours with control goal:

Answer 29

* Treatment goal directly treat * Generalization goal Related, expectation that these will be affected by training on treatment goal * Control goal Unrelated, don’t expect to change Controls for external factors (e.g., maturation) Difficult to choose ?Where can you randomize?

Question 30

Explain Multiple Baseline across subjects:

Answer 30

Single subject design repeated with similar participants Looking to replicate findings Replication increases internal validity Increases external validity/generalizability somewhat Non-concurrent (or staggered or variable) baseline important for internal validity

Question 31

Explain Multiple Baseline across settings:

Answer 31

* Single subject design repeated with same individual(s) in different settings * Typically used to show generalization – what you want in treatment Settings can be somewhat like generalization goal – expect to see delay/lesser impact but may see change * With some individuals, may predict that you need to treat in each setting – use as experimental control (don’t see change until treat in each setting)

Question 32

Explain alternating treatment designs:

Answer 32

* Theoretically, 2 treatments should have independent and differential effects * Including a second treatment clarifies the impact of each on the DV * A B C B C B etc. design: Compares the effects of rapidly alternating treatments B and C (where 1 of the treatments may be a placebo) - Problem 1: Treatment reactivity Must only compare adjacent phases C cannot be assessed separately from B Relationship between response to C and baseline (A) is unclear A B A C: Introduces a 2nd baseline: alleviates reactivity somewhat -Problem 2: Order effects Order effects might be addressed using an A B C A C B design Can be controlled through replication across subjects with alternate ordering

Question 33

How to reduce reactivity in alternating treatment designs?

Answer 33

- Sometimes treat one set of targets with B and another set with C to reduce reactivity - Alternate design Can see studies where both treatments done simultaneously Used different items in each treatment - Need to question whether effective for reactivity based on treatment and target

Question 34

Explain interaction designs:

Answer 34

* Purpose – to examine the impact of pieces of a complex intervention * Reduction Design Total package evaluated and then compare to single component A-BC-B-BC-B * Additive Design Determine effect of a simple protocol then add another component A-B-BC-B-BC * Allow you to look at interactions among components of treatment Order/reactivity effects are not eliminated Can be controlled through replication across subjects

Question 35

Explain Multiple Treatment designs:

Answer 35

* Interactive Designs A B BC (simplest): Allows analysis of combined (BC) as well as separate (B) treatment effects Variations on this theme: e.g., A BC B BC, etc… Can potentially isolate impact of individual treatment components * Like Interaction Designs but view treatments as distinct rather than 2 parts of a package

Question 36

What are 4 scientific methodology components?

Answer 36

Operational Specificity Repeated Measures Interobserver agreement External validity

Question 37

Explain operational specificity: (IV)

Answer 37

- IV (treatment) Instructions to participant Stimuli/materials Response expected Feedback/reinforcement Scheduling & amount of treatment Etc. - Essentially so could replicate - Treatment fidelity evidence

Question 38

Explain operational specificity: (DV)

Answer 38

- DV (outcome measure(s)) Trial scoring (% correct – need obligatory context) Rate measures Frequency measures Interval and Time sampling Must set criterion level of performance – when you’ll say goal was mastered More likely to evaluate outcomes in multiple ways than in group designs - Participant(s) characteristics - Interventionist characteristics - Setting characteristics

Question 39

How are repeated measures done? (3)

Answer 39

Done in a consistent way At regular intervals More frequently than in group designs

Question 39

What are the 2 types of analysis of Single-Subject Designs?

Answer 39

Visual Statistical

Question 40

What is a threat to internal validity from repeated measures?

Answer 40

Testing effect (subject gets better with time)

Question 41

What do you do in Visual Inspection when analyzing single-subject designs?

Answer 41

* Compare adjacent phases Variability: within each phase Level: value of dependent measure Last point of previous phase to 1st point of following Average of adjacent phases: misleading with high varibility or sharp slopes Trend: direction of change in a phase 3 patterns: Accelerating, decelerating, flat Slope: rate of change

Question 42

What is a Celeration line?

Answer 42

Compare slopes Compare overlap in data points Best fit lines