1
Q
What experimental design is not a true experiment and why?
A
- A quasi-experimental design
- lack randomization
- lack comparison groups
2
Q
What is the gold standard of true experimental design?
A
Randomized Control Trial
3
Q
What occurs in a true experimental design?
A
- Subjects are randomly assigned to at least 2 comparison groups
- Experiment enables control over most threats to internal validity and provides the strongest evidence for causal relationships
4
Q
What is a completely randomized design?
A
- Between subject design
* Subjects assigned to groups based on a randomization process
5
Q
What is a randomized block design?
A
- Subjects classified according to an attribute (blocking variable)
- Then randomized to treatment groups
6
Q
What is a repeated-measures design?
A
- Within-subjects design (everyone gets same interventions)
* Subject acts as own control
7
Q
How many independent variables do single-factor designs have?
A
*One independent variable
8
Q
How many independent variables do multi-factor designs have?
A
*Two or more independent variables
9
Q
What is a single-factor design?
A
- One way design
- 1 independent variable is investigated
- 1 or more dependent variables
- Looking at how many independent variables there are, not how many dependent variables there are
10
Q
What occurs with a RCT with 2 groups based on random assignment?
A
- Pretest-posttest control groups design
- Independent groups= treatment arms
- Testing pre and post treatment
- Changes in experimental group are attributable to the treatment
- Establishes cause and effect relationship
- Change in the experimental group is the post result minus the pre result to get the overall result. The change becomes the dependent variable
11
Q
What occurs in a 2-group pretest posttest design?
A
- Comparison group recieves a second form of the intervention
- 2 experimental groups formed by random assignment
- Control group is not feasible or ethical
- Compares new treatment with “standard care”
12
Q
What is a multi-group pretest posttest control group design?
A
- Multiple intervention groups
* Includes a control group
13
Q
What are pre-test post-test designs strong in?
A
*Strong in internal validity
*
14
Q
How can the initial equivalence of groups be established?
A
*By pretest scores (important for inferring causality)
15
Q
How is selection bias controlled in pretest posttest designs?
A
*Controlled because of random assignments
16
Q
What should effect groups equally in pretest posttest designs?
A
*History, maturation, testing, instrumentation
17
Q
How is the analysis of pretest posttest designs often done?
A
- Often analyzed using change scores
- difference between posttest and pretest
- Also can use analysis of covariance (ANCOVA) to compare posttest scores
- using pretest scores as covariate
18
Q
What is a posttest only control group design?
A
- Same as pre-test posttest control group design
- EXCEPT no pre-test
- Used when dependent variables can only be assessed following treatment
- e.g. length of stay in hospital (see example pg 199)
- Used when pretest is impractical or detrimental
- Is an experimental design involving randomization and comparison groups
- Strong internal validity
- Assumes groups are equivalent prior to treatment
- works best with large samples to increase probability of “equivalency”
19
Q
What occurs in a multi-factor design for independent groups?
A
- single factor designs have 1 independent variable (with 1 or more levels) and do not account for interactions of severable variables
- Multi-factor designs have 2 or more independent variables
20
Q
What occurs in a factorial design?
A
- Factorial design incorporates two or more independent variables, with subjects randomly assigned to various combinations of levels of the two variables
- Two-way (two-factor) design has 2 independent variables
- Three-way (three-factor) design has 3 independent variables
21
Q
What is a survey?
A
- A series of questions
- interview
- questionnaire (written/electronic)
- Can be used in:
- experimental studies, exploratory studies, descriptive studies
22
Q
What occurs in an interview?
A
- Ask questions and record answers
- Structured format
- Unstructured format
23
Q
What occurs in a structured interview?
A
- Standard set of questions
- Same questions in same order to all subjects
- Same response choices
24
Q
What occurs in an unstructured interview?
A
- Less formal
- Open ended
- Conversational
- Often used in qualitative studies
25
What occurs in questionnaires?
* Structured surveys
* Self-administered
* Computerized or pen/paper
* Efficient as completed on subject’s own time
* Reduced bias from interactions with an interviewer
* Disadvantage is the potential for misunderstanding or interpreting questions
* Mail, electronic distribution
* Low return rates (30-60%) limit external validity of results
26
What are data collected via interview or questionnaire based on?
* SELF REPORT!
- no direct observation by the researcher of subject's behavior
- potential for bias or inaccuracy
- recall bias
27
What should be asked when making a survey design?
* Delineate the overall research question
- What are the objectives (guiding questions) of the study?
- These objectives focus the content of the questionnaire
* Outline of the questionnaire (relate to objectives)
* Review existing instruments
- Can they be adapted for my study?
* Write questions that address each of the objectives
28
What do you ask when you distribute your preliminary draft of survey to colleagues?
* Ask for feedback
* Revise
* Distribute again
* Helps establish content validity of the instrument
29
What size sample should you do a pilot test on?
* Small sample of 5-10 research subjects
- interview respondents for feedback
- revise
- retest
30
How do you select a sample for surveys?
* All PTs in Michigan
| * Purchase mailing lists
31
How do you contact respondents?
* Cover letter
| * Follow Up
32
What do scales of surveys provide?
*Provide rating of degree to which subject possesses a characteristic/attitude/value
33
What are the scales of surveys?
* Likert:
- Strongly agree
- Agree
- Neutral
- Disagree
- Strongly disagree
* Likert scales coded 1-5
* Calculate overall score by adding answers
* 1 item does not carry more weight than others
* *visual analog scale
- Place a mark on the 100 mm line
34
What is the Delphi Technique?
* Experts complete multiple e.g. 3 rounds of questionnaires
* Researcher reviews and distributes findings after each round
* Eventually come to consensus on an issue
- e.g. what should entry level knowledge be for a particular topic
35
How do you do an analysis of survey data?
* Code the data e.g. male 1, female 0;
* Fear of falling 1, no fear 0
* SA=5; A=4; N=3; D=2; SD=1
36
What does descriptive statistics do?
* Summarize responses
* Mean age; years of education etc.
* Categorical data
- Frequency/percentages: 25 males (33% of sample)
- 40% SA; 30% A; 20% Neutral etc
* Scores on a scale may be summed Ex. mean ABC=84%
37
What does an IRB do?
* Must approve survey research
* Protection from psychological harm
* confidentiality
* Informed consent must be provided by participants
38
What is the best way for clinicians to seek evidence for interventions and assessments?
* Systematic Reviews
| - Cochrane Collaboration
39
What should a study question be?
* Question is very specific
| * Well described purpose statement
40
What is a narrative review?
*There's a broad question to be answered with search strategies and selection of articles not usually described. Appraisal is not always rigorous and the conclusions are usually descriptive/qualitative
41
What is a systematic review like?
*The question is focused and the search strategies/databases are often described in detail. The selection of articles are rigorous based on specific criteria. Appraisal is very rigorous and the conclusion maybe qualitative or quantivative (meta-analysis)
42
What are the selection criteria of a systematic review?
* “Subjects” of the review are the studies
* Specify inclusion/exclusion criteria
* Based on:
- Types of studies
- Types of participants
- Types of interventions
- Types of outcome measures
43
What is the search strategy for a systematic review?
* Select keywords
* Identify resources
- Databases
- Grey literature (unpublished studies)
* Publication bias
* Conduct the search and retrieve relevant papers
44
How do you evaluate quality of selected studies?
* Critical review
* Record on a form
* Evaluate quality of design and data analysis
45
What are the types of study bias?
* Selection bias
* Performance bias
* Attrition bias
* Detection bias
46
What is the Jadad Rating scale?
* 3 questions
* Was study randomized (1 point if yes)
* Was study described as double blind (1 pt yes)
* Was there a description of withdrawals and dropouts (1 pt yes)
47
How many items is the PEDro Scale?
*11 items
48
What is Data synthesis?
* Heterogeneity (dissimilarity) or homogeneity of the included studies (variability across studies)
- Composition of treatment groups (inclusion/exclusion criteria)
- Design of study (including length of follow up)
- Management of patients (treatments provided/Presence of complications)
49
What are the analysis/synthesis of findings?
* Overall conclusions based on quality of evidence obtained
| * Often summarize findings in a table
50
What is a forest plot?
* Represents the overall result of the meta-analysis
- square is the outcome for that study [relative risk (RR)]; size of square relates to weighting of study based on sample size
- line represents confidence interval (CI) around the RR
- diamond is combined overall estimate of results [includes pooled point estimate (center of diamond) and CI (horizontal tips of diamond)]
51
Describe the confidence interval (CI) of a forest plot?
*If a CI of a result crosses the line of no effect, then either a significant difference does not exist b/w the treatment and the control or the sample size was too small to show an effect
52
What is meta-analysis homogeneity?
*results of each individual trial are mathematically compatible with the results of the other trials
53
What are heterogeneous studies?
* If the CIs do not overlap; heterogeneous studies
| - no common treatment effect across the studies
54
What is a two-way factorial design?
* Incorporates 2 independent variables
- effect of intensity (vigorous/moderate) on exercise behavior
- Effect of location (home/community center) on exercise behavior
- 2x2 design means 2 independent variables and 2 levels of each independent variables (4 groups)
55
What are the two-way factorial design main effects?
* Is there an effect of moderate versus vigorous exercise
* Is there an effect of exercsing at home or in community
* This is examining main effect of each independent variable
56
What are two-way factorial design interactions?
* Can also examine interaction effects between 2 independent variables
- Effect of 1 variable varies at different levels of the second variable
- e.g. maybe moderate exercise is more effective in changing exercise behavior but only when performed at a community center
57
When is a randomized block design used?
* When there is concern that an extraneous factor such as gender might influence differences between groups
- build the variable into the design as an independent variable
58
How many independent groups are in a repeated measures design?
* Up to now considered 2 independent groups
| - experimental and control groups created by random assignment or by blocking
59
What is a repeated measures design?
* One group of subjects is tested under all conditions
* Each subject acts as their own control
* aka within-subjects design
60
What are the advantages to a repeated measures design?
* Subject differences can be controlled
- Ex: differences between experimental and control groups are nullified bc no groups used
- Physiological and other factors remain constant throughout the experiment
- Subjects act as own controls provides most equivalent "comparison group" possible
61
What are the disadvantages of repeated measures design?
* Learning/practice effects when one person repeats measurements over and over
* Carryover effects when exposed to multiple treatment conditions
- Must allow enough time for dissipation of previous effects so there’s not cross-over
62
Why might repeated measures designs not be considered true experiments?
* Bc no randomized comparison groups
- However if they incorporate randomization of the order of repeated treatments/interventions then can be considered experiment
63
What is a one-way repeated measures design?
*One group of subjects is exposed to all levels of one independent variable
64
What is Order Effects?
*were responses dependent on which condition preceded which other condition?
65
What is the Solution to problem of order effects?
*Randomize order of conditions/interventions for each subject so there is no bias in choosing order of testing
66
What is a two-way design with two repeated measures?
* 2 repeated measures (= 2 independent variables)
- Type of lift (squat/stoop)
- Orthosis (yes/no)
* Each person exposed to 4 test conditions
- 2-way design (2x2 design)
67
What is a Mixed design for multi-factor repeated measure design?
* 2 independent variables
- Exercise is independent factor
- -2 independent groups (experimental; control)
- Time is repeated factor
- -3 time periods (pretest; time period 1; time period 2)
* 2-way design (2 x 3 design)
68
What do Quasi-experimental designs lack compared to experimental designs?
* Random assignment
* Comparison group
* Or both
69
What are quasi-experimental designs?
* May involve non-equivalent groups
* May be a reasonable alternative to RCT
* Conclusions drawn must take into account biases of the sample
70
What occurs in a one-group pretest-posttest design?
* Effect of treatment is determined by change in pre- and posttest scores
* Pretest---intervention---posttest
* Vulnerable to threats to internal validity because no control group
- History
- Maturation
- Testing
71
What is a one-way repeated measure design over time?
* Effect of treatment over time
* Pretest-intervention-posttest 1- posttest 2
* no control group so internal validity threatened
72
What is a Non-equivalent pretest-posttest control group design?
* Similar to pretest-posttest experimental design EXCEPT subjects not assigned to groups randomly e.g. volunteers self-select groups
* EXP: Pretest--Intervention--Posttest
* CON: pretest--no intervention--posttest
73
What is a non-equivalent posttest only control group design?
* Static group comparison
* EXP: intervention --Posttest
* CON: no intervention --Posttest
74
What is a single subject design?
* Draw conclusion on treatment effects based on 1 patient’s response
* Controlled experimental approach
* Independent variable is treatment
* Dependent variable is target behavior (outcome)
* Also called N of 1 study or Time series designs
75
What is the Structure of a single-subject study?
* Repeated Measurements
- Each session; observe trends
* At least 2 testing phases:
- Baseline (A phase)
- Intervention
* Target behavior is measured across both phases on multiple occasions
* Baseline phase: state of target behavior over time in the absence of treatment (control conditions)
76
In a single subject study, what occurs when treatment starts when talking about change?
* When treatment starts, any change from baseline to intervention phase is attributed to the intervention
* Baseline data
- comparison for evaluating potential cause and effect relationship between intervention and target behavior
* Baseline period = A
* Intervention period = B
* A-B design
77
How do single subject designs differ from traditional experimental designs?
*Multiple assessments in baseline and intervention phases
78
What are the ethical issues regarding baseline conditions with single subject designs?
*Withholding treatment
79
What are baseline characteristics?
* 2 baseline data characteristics are important for interpreting clinical outcomes
- Stability (consistency of response over time): stable or variable
- Trend: accelerating or decelerating
80
What is the length of phases?
* Best to have equal phase length
* Often 1 week per phase
- Daily measurements
- Minimum of 3-4 measurements per phase to detect trends
* Greater number of data points easier it will be to identify trends
* Often measures can be taken more frequently than daily if behavior changes rapidly
- More than a single session
81
What are Target Behaviors?
* Choose clinically relevant outcomes measures for a particular patient
- Strength
- ROM
- Gait speed
- Balance measures
- Pain
82
What are the limitations of A-B Design?
* Experiments can control for threats to internal validity
* To do this in the A-B single subject design is more challenging
- Other treatments/events (history effects)
* What other evidence can we include to strengthen design control ?
- to increase confidence that treatment caused the changes in target behavior
83
What is additional Control?
* *Replication of effects
* Repeat phases
- Withdrawal designs—treatment: no treatment
- Withdrawing and reinstating baseline and treatment conditions
- Withdraw intervention and show that target behavior occurs only in presence of treatment
- -2nd baseline period (A-B-A design)
- -Could also include a 2nd intervention phase (A-B-A-B design—see over)
84
What is Data analysis- Visual?
* Visual
- Level (last data point of a phase to first data point of next phase)
- Trend (direction of change in a phase)
- -Accelerating or decelerating
- -Slope of a trend (rate of change in the data)
85
What is a Data Analysis- Celebration line used for?
* Used to estimate trends in the data points
| * AKA "split middle line" as it divides the data points in a phase into 2 halves
86
What is Generalization?
* Single subject research can provide data for clinical decision making
* Not enough to show effect during intervention period on a single patient
- Must also be able to show changes in the target behavior will occur in other individuals
* Generalization: external validity for the single case
* Assume treatment will be effective in others with similar characteristics
87
What is an observational design?
* No manipulation of variables as in experimental designs
| * Exploratory or descriptive
88
What is exploratory research?
* Systematic investigation of relationships among variables
- e.g. association of leg weakness and falls
* Not used to establish cause-and-effect relationships b/w variables
89
How can you conduct exploratory research?
*Retrospectively or Prospectively
90
What is Prospective?
Variables measured in the present and follow subjects in a study
**More reliable than retrospective studies
91
What is Retrospective?
* Use of data that have already been collected
- medical records, databases
* Researcher can't control data collection methods
92
What occurs in Longitudinal Research?
* Follow a cohort over time taking repeated measurements
* Can observe growth and change in individuals over time
* Often involve large cohorts followed over long periods of time
- E.g. Framingham Heart Study (been going 40-50 yrs)
* Threats to internal validity relate to
- repeated testing
- attrition
93
What occurs in Cross-sectional research?
* Gather data as a “snap shot” in time
* Very efficient
* All subjects tested more or less at same time
94
What is a correlational study?
* Foundation of exploratory studies is process of correlation (degree of association)
- Covariation in data (extent to which one variable varies with another variable)
* Look at several variables at once to see which are related
* can make predictions (predictive correlational study) based on observed relationships between variables
- -Cholesterol level: age, diet, gender, genetics, etc
- -Regression (stats procedure)
95
What is the purpose of a correlational study?
*Purpose is to describe the nature of existing relationship among variables
96
What is a case-control study?
* Retrospective
* not randomized
* Purpose of a case-control study is to determine if the frequency of an exposure (e.g. poor nutrition; smoking) is different in cases and controls
* Choice of controls is critical
- match cases and controls for age, gender, SES, etc.
97
What is the selection bias in case-control studies?
* Selection bias: choose cases and controls regardless of exposure history
* Beware of misclassification ie cases and controls
98
What is observation bias in a case control study?
*difference in the way information about disease or exposure is obtained from the groups
99
What is interviewer bias in a case control study?
*Person collecting data elicits, records or interprets info differently from cases and controls
100
What is recall bias in a case control study?
*Subjects remember exposure differently
101
What is a cohort study?
* Follow a group(s) overtime (prospective)
* Group 1: Exercisers
* Group 2: Sedentary (matched)
* Not randomized
102
What is Causality in Observational studies?
* RCT (experimental): cause-and-effect relationships
* Case control and cohort studies do not involve experiments or manipulation of variables
- Causation (cause and effect i.e. did the exposure cause disease) is established by other methods
103
What is Causality?
* Establish a time sequence: exposure precedes disease
* Strength of association: relative risk
* Biologic credibility
* Consistency with other studies
* Dose-response relationship-
104
What is a methodological study?
* Type of exploratory study
| * Use correlational methods to examine reliability and validity of measuring instruments
105
What is a historical study?
* Type of exploratory study
* A historical study reconstructs the past on the basis of archives and other records to suggest relationships of historical interest to a discipline
106
What populations does descriptive research describe?
* Characteristics
* Behaviors
* Conditions
107
What does Descriptive research involve?
* May involve prospective or retrospective data collection
* Design may be longitudinal or cross-sectional
* Surveys and secondary analyses of clinical databases often used as data sources for analysis
108
What are the categories of descriptive research?
* Developmental research
* Normative studies
* Qualitative research
* Descriptive surveys
* Case studies / case reports
109
What does developmental research involve?
* Involves description of developmental change and sequencing of behaviors in people over time
- motor development in children
- lifespan
110
What do longitudinal methods of developmental research involve?
*Longitudinal methods involve collecting data over time—focus on natural history of a disease (eg CMT study)
111
How can you use a cross-sectional study in developmental research?
*Can also use cross-sectional methods and study different age groups at a point in time
112
What is a normative study?
* Purpose is to describe typical or standard values for characteristics of a population
* Describe norms as a mean and a range of acceptable values
* Norms are used as a basis for prescribing interventions
113
What is qualitative research?
* Describes how individuals perceive their own experiences within a specific social context
- What it means to live with a spinal cord injury
* Helps us understand the patient’s view of the world
- Important in designing interventions
* Data collected by interviews and observation
- Participant observation (researcher embedded in the group)
- Field observation: nonparticipant
114
What are descriptive surveys?
* Often used as a source of data to collect information about a specific group
- To describe their characteristics, or risk for disease, or other attributes
115
What occurs in case studies?
* Important for developing a clinical knowledge base
* In-depth description of a person’s condition or response to treatment
* Case series involves observations in a number of similar cases
* Often involve unusual diagnoses that are challenging
* May highlight avenues for future research
116
What is the case studies format?
* Comprehensive description of the subjects background, present status, and responses to intervention
* Poses questions for further study
117
What does the introduction do in a case studies format?
*Describes background literature to the pts problem
118
what does the pt history do in a case studies format?
*Problems, symptoms, prior treatments, demographic and social info
119
What are the results in a case studies format?
*pts response and any follow-up data
120
What is the discussion in a case studies format?
*Interpretation of outcomes and conclusions
121
What does a case studies format consist of?
* introduciton
* pt history
* treatment plan
* Results
* Discussion
122
How do you apply literature to patients?
* Systematic Review
* Critically Appraised Topic (CAT)
- -Brief Summary of a search and critical appraisal of literature on a clinical question
- -Standardized format
- -Provides statement of clinical relevance
123
What is a CAT?
* Initiated by a patient encounter usually due to gap in knowledge
* Search for and appraises best evidence
* Summarizes evidence
* Integrates evidence with clinical expertise
* Suggest how information can be applied to practice
* Usually 1-2 pages
124
How are CATs and systematic reviews similar?
*They both have the goal of applying their information/findings to people
125
What is the format of a CAT?
* Title
* Author/date
* Clinical scenario (description of case that prompted the question)
* Clinical question (PICO format)
* Clinical bottom line (summary of how results can be applied)
* Search history/strategy
* Citations
* Summary of the study/ies (design; sample; intervention; outcome measures; data analysis)
* Summary of the evidence (results summarized)
* Critical comments on the study (internal/external and statistical validity of the study)
126
When do you use CATs?
* Useful at point of care
* Can be created out of case conferences
* “CAT Banks” established by institutions
127
What are some of the limitations of a CAT?
* Limited shelf life as new evidence becomes available
* Not as rigorous as a systematic review
- -1-2 references and do not represent full scope of the literature on a topic
Critical Inquiry
flashcards
Decks in class (3)
# Cards
