1
Q
Focus of Behavior Analysis
A
- Primary focus of Bx-analytic assessment: Identify prob with bx (excesses & skill deficits)
- Also need to measure & assess: Effects of bx analyst’s involvement in treatment
- Social Validity
- Risk
- Treatment integrity
2
Q
Review: Behavioral Assessment
A
*Systematic gathering of info to make data-based decisions regarding behavior & the environment
3
Q
Behavior - Analytic Assessment
A
- Purpose is to Gain specific info about individual performance within a specific env
- Not diagnostic
- Does not necessarily result in a global score or grade level equivalent
4
Q
Different Approaches: Results of Assessments
A
- Behavioral:
- Leads to specific targets
- Educational/ psychological:
- Leads to Broad “Level” or global score
5
Q
Different Approaches:
Combined or Separated Operants
A
- Behavioral:
- Separates out each operant
- Educational/Psychological:
- Often combines operants
6
Q
Different Approaches Freq of Assessments
A
- Behavioral:
- Ongoing
- Educational/ Psychological:
- Snapshot
7
Q
Different Approaches:
Types of Data Used
A
- Behavioral:
- Requires direct (objective) observation of bx
- Educational/ psychological:
- (Subjective) self-report maybe only measure
8
Q
Which if the following are features of a behavioral assessment?
a. Ongoing
b. Combines operants
c. Provides a global score
d. Separates by operant
e. Requires observation
f. Self-report can be sufficient
A
a. Ongoing
d. Separates by operant
e. Requires observation
9
Q
Objective Measures
A
- Measures that are impartial & independently verifiable
- e.g: The coffee is 160 F
10
Q
Subjective Measures
A
- Measures that are influenced by personal opinions & can only be reported by the individual
- E.g: The coffee is too cold
11
Q
Preference For Objective Measures in Bx-Analytic Assessment
A
- Problems with subjective measure:
- Can change over time (not reliable)
- Not casual
- Counter to the natural science approach
- Behavior-Analytic assessments will always include objective measures
- May also supplement with subjective
12
Q
Which type of measures are independently verifiable?
A. Objective
B. Subjective
A
A. Objective measures are independently verifiable while subjective measures are influenced by personal opinions and are susceptible to biases.
13
Q
Rating a product as 4 out of 5 stars would be considered what type of measure?
A. Objective
B. Subjective
A
B. Subjective
Whether a product is rated as 4/5 stars is based on a person’s opinion of that product. It is not clear what criterion the person might use to rate the product that way. Therefore, this would be a subjective measure.
14
Q
Write an example of an objective measure:
A
-“Completed work in 3 hours.” Or “Asked for help 4 times.”
An objective measure can be independently verified. Any precise measure based on direct observation would be a good example of an objective measure.
15
Q
Wolf (1978) on Social Validity
A
“Social Validity” :The Case for Subjective Measurement or How Applied Bx Analysis if Finding Its Heart”
16
Q
Social Validity
A
*The degree to which the consumer finds our involvement acceptable from beginning to end
17
Q
Importance of Social Validity
A
- Consumer is more likely to stick with the treatment
- Better outcomes
- Encourage others to seek treatment/ help
18
Q
Poor Social Validity
A
- AKA social invalidity
- Actions the consumer takes when our involvement is not acceptable
- Unsatisfied consumers may:
- Not implement parts of program
- Withdraw
- Share disapproval with others
19
Q
Case for Subjective Measures
A
*JABA’s Purpose: “Publication of applications of the analysis of bx to problems of social importance”
20
Q
Laboratory and Applied Settings
A
- In Lab Settings:
- Select species based on genetics
- Can control individual history
- In applied settings:
- Cannot control for species
- Not using novel bx
21
Q
Subjectivity in Social Validity
A
- Cannot rely on subjective measures alone
- Not sufficient, but necessary
- Standard practice
22
Q
Which is the best description of social validity?
a. How certain we are that the procedures caused the change
b. The extent to which consumers find service acceptable
c. The extent to which a procedure affects society at large
A
b. The extent to which consumers find service acceptable
23
Q
When assessing social validity, the use of subjective measures may be appropriate.
a. True
b. False
A
a. True
24
Q
Social Validity Assessment
A
- Examination of the acceptability, Viability & Appropriateness of a client goal, programmed intervention, and/or outcome
- Are the changes in bx of clinical or applied importance?
25
Areas to Measure Social Validity
- Goal
- Procedures
- Results
26
Social Validity Goals
* Are the specific behavioral Goals really what society want?"
- The consumer
- The important people in the consumer's life
- The society
27
Uses Social Validity
* Assess throughout treatment
* Remember:
- Measures must be high in each area
- Measures maybe Subjective
* Be responsive to the results of assessment
28
In which areas do we assess social validity?
a. Goal
b. Rapport
c. Attire
d. Procedures
e. Results
f. Demeanor
a. Goal
d. Procedures
e. Results
29
Are the specific behavioral ________ really what society wants?
-Goals
We assess social validity at all stages of the provision of behavior-analytic services: Before treatment (goals), during treatment (procedures), and after treatment (results).
30
Do consumers consider the treatment _______ acceptable?
-Procedures
We assess social validity at all stages of the provision of bx-analytic services: Before treatment (goals), during treatment (procedures), and after treatment (results).
31
Are consumers satisfied with the ________?
-Results
We assess social validity at all stages of the provision of bx-analytic services: Before treatment (goals), during treatment (procedures), and after treatment (results).
32
Prevalence of Social Validity Measures in Research
* Review of journals reporting social validity in studies:
| - Carr et al (1999)
33
Prevalence: Take-Home Point
* Research ques often focus on some other primary phenomenon
* Prevalence in practice is still an empirical ques
34
Importance of Social Validity Measures in Applied Practice
*"Efficacy without adequate social acceptability is a recipe for marginalization
35
Why is conducting social validity assessments increasingly important in clinical practice?
a. Clinical treatment does not require IRB approval
b. Treatment often only applies one procedure at a time
c. Results of treatment typically are sufficient to satisfy consumers
a. Clinical treatment does not require IRB approval
36
What could result from carrying out interventions that are not socially valid?
a. Clients do not make progress
b. Caregivers may discontinue services
c. Other people who could benefit might not seek help
d. All of these
d. All of these
37
Early Decisions in Service Delivery
* Selecting goals & target
- Social Validity
* Treatment needs
- Intensity of services
- Involvement from others
38
Determining Social Validity of Goals
*Are the specific behavioral goals really what society wants?"
39
Applied Dimension of ABA
"In behavioral application, the bx, stimuli and/ or organism under study are chosen because of their importance to man & society"
40
Social Validity & the Applied Dimension E.g. 1
- E.g: Addressing feeding with a child with an extremely limited diet
- Non-e.g: Teaching a child a type when they cannot read
41
E.g. of Socially Valid Goals
- Live in my own apartment
- Graduate high school
- Work in a restaurant
- Communicate in a way to express needs
42
Non-examples of Socially Valid Goals
-Strict compliance to authority
May result in less autonomy
-Toddler never whines
Unreasonable expectation
43
The applied dimension of ABA tells us that goals:
a. Should be large enough to make a difference
b. Should demonstrate experimental control
c. Are chosen for their social importance
d. Generalize across contexts
c. Are chosen for their social importance
44
Which of the following are the most likely to be socially valid goals?
(check all that apply)
a. Asks for help when needed
b. Can identify font types by sight
c. Completes morning hygiene routine independently
d. Can arrange names of former vice president of the U.S. by year
a. Asks for help when needed
| c. Completes morning hygiene routine independently
45
3 Levels of Goals
Assess social validity at these levels:
- Broad social goals
- Behavioral categories
- Discrete responses
46
Broad Social Goals
* Ultimate outcomes
- Long-term goals
* E.g: Live independently
* Non-e.g.
- Self-actualize
* Not Valid = Maintain a healthy lifestyle as defined by recent fad
* Valid: Maintain a healthy lifestyle
47
Behavioral Categories
* Intermediate outcomes
- e.g:
- Hygiene
- Time management
- Meal preparation
* Not valid: Consume ONLY "healthy" food
* Valid = Eat a balanced, healthy diet
48
Discrete Responses
* Specific target bx
- E.g:
- Showering
- Setting alarm clock
- Bringing lunch to work
* Not Valid: Consumes foods that are not in agreement with religious beliefs
* Valid - Eat 7 servings of fruits & veg per day
49
Selecting Goals With the Client
* Child at library, common play area
* Language delays
* Prob bx when other children approach/ touch associated items
50
Selecting Goals With the Client
* Child @ library, common play area
* Language delays
* Prob bx when other children approach or touch associated items
* Do we teach:
- Ask to play alone?
- Tolerate presence of peers?
51
What are the three levels that are associated with social validity of goals?
a. Behavioral categories
b. Components of treatment prog
c. Discrete responses
d. Broad social goals
e. Proximal results
a. Behavioral categories
c. Discrete responses
d. Broad social goals
52
When assessing the social validity of goals:
a. Focus on ensuring validity at the discrete-responses level above all else
b. Favorable results in 2 to 3 areas is sufficient
c. Ensure validity at all levels
c. Ensure validity at all levels
53
What was the problem with the social skills training program in the story that Jose told?
a. The procedures used were not bx-analytic
b. The procedure used was not effective
c. The goals selected let to unintended results
c. The goals selected let to unintended results
54
What would have been a more socially valid behavior for the behavior analyst to target with the boy?
Walk away, report to a teacher, or teach conversational skills about topics of common interest
Jose provided some examples of alternative behavior that could have been more effective in escaping or avoiding the bully. They included making light of the situation or walking away.
55
General Process for Evaluating Social Validity
1. Identify who will be evaluating
2. Determine when to assess
3. Identify what to assess
4. Select methods to assess
5. Identify who will conduct the assessment
6. Collect & summarize the data
7. Reconcile any low scores
56
Identify Who Will Evaluate
- Direct consumers
- Indirect consumers
- Members of the immediate community
- Members of the extended community
57
Determine When to Assess Goals
* Pre-intervention:
- Selection of the goal
- Criterion for goal achievement
* Post-intervention:
- Determine if goals led to outcome
58
Identify What to Assess
- Select subjective and/ or objective measures of the goals
- Some e.g:
- "Importance of the goal
- Current client performance compared to the goal
59
Methods for Assessing Social Validity of Goals
* Indirect techniques
- Interviews/ Questionnaires
- Rating scales
* Direct techniques
- Normative comparison
- Norm-referenced assessment or curricula
60
Who could be relevant to include in social validity assessments?
a. Direct consumers
b. Family members
c. Community members
d. All of these
e. A and B but not C
d. All of these
61
Which types of techniques can be used to assess social validity?
a. Indirect
b. Direct
c. Both
c. Both
Both indirect and direct social validity assessment techniques can be used. An example of the former would be an interview or rating scale. An example of the latter would be making social comparisons with “competent” peers.
62
Indirect Techniques for Assessing the Social Validity of Goals
* Purpose:
- Evaluation of stakeholder values
* Assessments used:
- Interviews, questionnaires, rating scales
* Schedule for assessment:
- Pre-intervention
63
Review: Behavioral Interview
*A series of ques posed by a Bx analyst to someone familiar with the bx of the individual service recipient
64
Purpose of interviews for Assessing Social Validity of Goals
* Prioritizing goals
* Identifying expectations of bx
- Client's expectations for self
- Caregiver expectations
* Identifying values of the individuals & community
65
Sample Ques to ask for Assessing Social Validity of Goals
"If you could change 1 thing today that would make _______'s life better tomorrow, what would it be?
- "What is it about _____ that keeps you up at night?"
- "What's imp to you?"
- Focus on the Client's benefit
66
What is the purpose of interviews when assessing the social validity of goals?
a. Help prioritize potential targets
b. Ensure accurate procedures
c. Guarantee adherence to treatment
d. Ensure caregivers accept the targets you select
a. Help prioritize potential targets
The purpose of interviews when assessing the social validity of goals is to help prioritize potential targets. The accuracy of the procedures and adherence of treatment are related to treatment integrity, not social validity. Caregivers accepting the targets you select is not necessarily related to social validity since they might accept a target that has no major benefit to the client.
67
Review: Rating Scales
*Structured ques that use a Likert scale to gather info about bx & the env during a behavioral assessment
68
Purpose of Rating Scales for Assessing Social Validity of Goals
* Attach a # to subjective measures
| * Helpful when there is disagreement about potential goals
69
Prioritizing Goals
* A learner may have multiple problems with Bx (skill deficits & or behavioral excesses)
- Cannot address them all simultaneously
- Different caregivers or professionals may disagree about priorities
* Use rating system
70
Rating Scales for Prioritizing Goals
* List potential target bxs
* Have team members rate each goal
- Client
- Parent
- Teacher
- Others
71
How can rating scales help in prioritizing goals?
a. Help identify satisfaction after goals have been determined
b. Can provide evidence that the goal you have already selected is indeed the best
c. Can help sort an acceptable sequence of targets
c. Can help sort an acceptable sequence of targets
We can use rating scales to help us select and prioritize goals—we don’t use them once goals have already been determined. Rating scales do not provide evidence a selected goal is the best because rating scales are subjective. Rating scales can help sort an acceptable sequence of targets by prioritizing goals.
72
The results of rating scales are considered:
a. Direct measures
b. Indirect measures
b. Indirect measures
The results of rating scales are considered indirect measures because they are based on opinion and people can use different criteria to select the importance of goals.
73
Direct Techniques for the social Validity of Goals
* Purpose: Identify bx of clients compared to others
* Assessments used: Normative comparisons & Norm-referenced assessment &/ or curricula
* Schedule for assessment:
- Pre-intervention
74
Normative Comparisons for Assessing Social Validity of Goals
* Collect data on target bx
- Of Client
- Of normative sample
* Compare data btw client & sample
* Set client goals
75
e.g. of Normative Comparison
* Goal: Student stays in his seat while in the classroom
- What is normal for the classroom?
- What is normal for the age?
- Does this impact academic performance?
76
Norm-References Assessment for Assessing Social Validity of Goals
* Established curricula
- Identify skill areas
- Level of complexity
- E.g:
- VB-MAPP
- Essential for Living
77
Normative comparison would likely be best used for which situation?
a. To help set the duration of brushing teeth for an adult
b. To determine how long to keep an arm still for a blood draw
c. To identify the appropriate speed for an employee bagging groceries
c. To identify the appropriate speed for an employee bagging groceries
78
Early Decisions in Service Delivery
* Selecting goals & targets
- Social validity
* Treatment needs
- Intensity of services
- Involvement from others
79
Intensity (Dosage) of Services
* # of direct service hours per week
| * Does not include supervision, programming, parent training etc.
80
Factors in Determining intensity of Services
- Severity of behavioral issue(s)
- Response to prior treatment
- Settings for training & generalization
81
Severity of Problem(s)
* How significant is the delay?
- More services provided for greater delays
* How intense is the challenging bx?
- More services provided when there is higher risk of harm
82
Response to Prior Treatment
* Has client received services in the past?
- If so how many hours?
* How quickly did bx change?
- If change is slow, more hours maybe required
83
Settings for Training & Generalization
* Does prob bx occurring multiple settings (home, school, work etc)
- More settings may require more hours
84
CASP Guidelines
| -The Council of Autism Service Providers
* ASD treatment guidelines written for funding sources
- Based on client need
* Intensity of services:
- Comprehensive (30-40 hrs per week)
- Focused (10-25 hrs per week)
85
Involvement From Others
* Stakeholders that interact with the client (family, teachers, staff)
* Identify need for specialist involvement
86
What factors should be considered when determining the intensity of services?
a. Number of technician available
b. Typical authorized hours
c. Severity of delay
d. All of these
c. Severity of delay
87
Assessments of Treatment Procedures
* Prior to selecting treatment procedures:
- Conduct risk-benefit analysis
- Assess social validity of procedures
* While implementing treatment procedures:
- Assess treatment integrity
88
Risk-Benefit Analysis
* Comparison of the risk of a situation to its related benefits"
* Goals:
- To be upfront with consumers
- Provide clarity to practitioners
89
Steps for Assessing Risks and Benefits
1. Assess the General risk factors for behavioral treatment
2. Assess the benefits of behavioral treatment
3. Assess the Risk factors for each behavioral procedure
4. Reconcile the risks & benefits with the key parties involved
90
General Risk Factors
- Nature of Bx
- Sufficient personnel
- Is the mediator well trained?
- Is the setting appropriate?
- Is the bx analyst experienced?
- Risk to other?
- Buy-in?
- Personal liability?
91
Nature of the Behavior
- Topography
- Intensity
- History
- Dangerous to self or others
- Destructive
- Disruptive
92
Sufficient Personnel
* Mediators in the client's env
- Direct care staff
- Parents
- Relevant family
- Teacher
* Caregiver/ consumer ratio
* Is it reasonable to expect them to carry out the procedures?
93
Prior Training of Mediator
- Has training been sufficient?
| - Can they be trained?
94
Appropriate Setting
* Is the setting safe?
- Client
- Mediator
- Other ppl
95
Experience of Bx Analyst
* Is this within your competency?
- Education
- Training
- Supervised experience
96
When should a risk-benefit analysis be conducted?
a. Only before treatment is started
b. Only if treatment may need to be changed
c. Both before treatment is started & if treatment requires change
c. Both before treatment is started & if treatment requires change
97
Identifying the number of people involved in treatment and whether there are enough people would address which risk factor?
a. Prior training of mediator
b. Sufficient personnel
c. Appropriate setting
d. Experience of bx analyst
b. Sufficient personnel
98
Risk to Others
*Does treatment present a risk to others in the setting?
-E.g.
Extinction burst
Emotional responding
Peers left unattended
*Do not downplay to gain approval
99
Client and Stakeholder Buy-in
* Approval & willingness to ensure the plan is implemented
* Might extend beyond direct implementers
- E.g. school admin, Counselors, Social workers etc
100
Liability of the Behavior Analyst
* The bx analyst assumes responsibility, even if others carry out procedures
* Consider
- Supervision
- Training
- Emergency procedures
101
Sheryl is identifying if support staff are willing to carry out the program as designed. She is assessing which risk factor?
a. Appropriate setting
b. Risk to others
c. Stakeholder buy-in
d. Liability to the bx analyst
c. Stakeholder buy-in
102
When possible, rephrase potentially uncomfortable procedures to improve acceptability if they are likely to be effective.
a. True
b. False
b. False
103
Benefits of Behavioral Treatment
* Benefits to:
- Client directly
- Mediators & caregivers
- Peers in setting
- General liability
- Setting
104
Benefit to Client
* Be specific about the expected benefits?
- New bx displayed e.g. child follows instructions
* Challenging bx reduced e.g. Running away eliminated, No aggressive threats
105
Benefits to Setting
*In classroom, Workshop or home general "atmosphere" may improve
106
Benefits to Caregivers
* Improving the situation often benefits mediators
| - Not the primary reason for change, but still significant!
107
Benefits to Peers in Setting
*Improvement for the client may result in benefits to others
-E.g.
'Added source of Rx
'Previously target of aggression
'Reduced disruptions
'Increased available attention
108
Benefits to General Liability
*Can reduce the likelihood of legal action
109
In considering the benefits to others in the immediate area:
a. These benefits take equal importance with direct benefits to the consumer
b. These benefits are considered secondary to the direct benefits
c. Only assess if the primary benefits do not appear great
b. These benefits are considered secondary to the direct benefits
Behavior analysts’ main responsibility is to the person directly receiving services. Therefore, benefits to others would be considered secondary.
110
Even though benefits to others in the immediate area do not directly benefit the client, these are included in a risk-benefit analysis.
a. True
b. False
a. True
Although the main responsibility is to the direct recipient of services, behavior analysts have a responsibility to all parties affected by services. As a result, we must consider the effect on direct and indirect consumers when developing interventions.
111
Assess Risk Factors
* Examine any proposed procedures
* List potential risks & benefits
* Consider client & others in env
- Caregivers/ mediators
- Peers
- General community
112
Risks and Benefits
- Of intervening
- Of not intervening
- Of alternative treatment
113
Reconcile With Stakeholders
* Generate a Report (If needed)
- Specific risks
- General risks
- Expected benefits
* Involved parties decide
* Reevaluates as needed
114
A single risk-benefit analysis should include all procedures.
a. True
b. False
b. False
A separate risk-benefit analysis should be conducted for each procedure
115
Social Validity of Procedures
* Acceptability of treatment procedures
| - "Dp the participants, caregivers & other consumers consider the treatment procedures acceptable"?
116
E.g. of Analyzing Social Validity at All Levels
* Goal: Learns to brush teeth independently (Socially Valid)
* Procedure: Hold head down & forcible brush teeth if client resists (NOT socially valid)
* Outcome/ result: Improved dental hygiene & more independence (Socially Valid)
117
Why conduct social validity assessments of procedures?
a. Interventions might otherwise be ineffective without assessment
b. Clients might bot be happy with procedures unless they see results
c. Questionable procedures may yield an entire program unacceptable, even with accepted goals & results
c. Questionable procedures may yield an entire program unacceptable, even with accepted goals & results
A problem with not conducting a social validity assessment is finding resistance to treatment when questionable procedures are recommended. In this case the issue is lack of cooperation with treatment, not necessarily that treatment is effective (if it were to be implemented accurately).
118
3 Levels of Procedures
| Assess social validity at these levels:
- Comprehensive treatment programs
- Components of a prog/ package
- Specific behavioral procedures
119
Comprehensive Treatment Program
* Clear description of the Full program
- E.gs
- Strictly ABA procedures (bx reduction & skill acquisition)
- Setting where services are provided (home, community etc)
- Parent training (Is it included?)
120
Components of a Treatment Program
* Treatment packages or general instructional approach used in the program
- E.g.
- Behavior improvement plan
- Discrete trial training
- Goal sequencing
- Parent training (How is it conducted?)
121
Specific Behavioral Procedures
* Planned treatment component
- E.g:
- Reinforcers
- Extinction
- Prompting
122
Assessing social validity of procedures should
a. Focus on the most frequently used procedure
b. Assess the most intrusive procedure only
c. Assess the entire program
c. Assess the entire program
Assessing social validity of procedures should focus on the entire program—not just the most frequently used or most intrusive procedures.
123
E.g. Assessing Social Validity of Procedures at All Levels
* Comprehensive Tx prog
- Valid = In-center therapy
- Not Valid = Lack of needed parent training
* Components of a Tx program
- Valid = Discrete trial teaching at table
- Not valid = Not enough natural env teaching
* Specific behavioral procedures within the Tx program
- Not Valid = Using large amounts of snack foods as Rx
- Valid = Using a variety of Rx
124
A parent of a client has provided feedback on in-home services. She is pleased overall with the program, including specific teaching procedures. However, she expressed concerns with lack of social skills training. What would be an appropriate course of action for the behavior analyst?
a. Continue with program as designed, as 2 of 3 areas are satisfactory
b. Discuss potential social skills training procedures & means to include them
c. Redesign the entire program from scratch
b. Discuss potential social skills training procedures & means to include them
125
General Process for Evaluating Social Validity
1. Identify who will be evaluating
2. Determine when to assess
3. Identify what to assess
4. Select methods & tools
5. Identify who will conduct the assessment
6. Collect & summarize the data
7. Reconcile any low scores
126
Methods for Assessing the Social Validity of Procedures
* Indirect techniques
- Interviews
- Questionnaires
- Rating scales
* Direct techniques
- Participant preference
- Sustained use
127
Indirect Techniques for the Social Validity of Procedures
* Purpose: Identify opinion about Feasibility & appropriateness of procedures
* Assessments used: Interviews, questionnaires, Rating scales
* Schedule for assessment: Pre-intervention & post-intervention
128
A rating scale asking about the potential danger of behavior is focusing on the social validity of_____.
-Goals
129
A rating scale asking about acceptable treatment steps is focusing on the social validity of ______.
-Procedures
130
Concerns About Subjectivity With Indirect Techniques
- Is it accurate or are they being polite
- Does it leave silent unsatisfied customers?
- One solution: Have a team member not involved in implementation conduct the assessment
131
Interviewing Direct Consumers for Social Validity of Procedures
* Consumers have been unlikely to be asked their opinion if they have limited language skills
* Dies not mean we can it involve them in the assessment process
132
Indices of Happiness
* A way to measure Bx that is correlated with pleasant opinions & feelings
* Can be used to assess social validity of procedures for learners with limited Verbal bx
- Display of these bxs may indicate client approval of procedures
133
Identifying Indices of Happiness
* Caregivers/ informants list:
- Signs the client is enjoying an activity
- Signs of displeasure
- Activities
* Look for agreement
* Provide choice
134
Measuring indices of happiness can provide direct measures of enjoyment.
a. True
b. False
b. False
135
How are measures of indices of happiness used to assess social validity?
These behavioral correlates may indicate that a client is enjoying the treatment/intervention when that client is otherwise unable to tell us whether s/he likes it or not
“Indices of happiness” should be used to assess social validity when the person is unable to express approval of procedures in a clear way (e.g., when the person is nonverbal).
136
Assessing Social Validity of Procedures: Bedtime Pass Study
* Prob: Bedtime resistance
* Typical treatment: Extinction
- Highly effective when implemented correctly
- Not a socially valid procedure (parents don't like it)
137
*Bedtime Pass Study
*Solution: Bedtime pass procedure
*Goals of procedure:
-Child going to bed & falling asleep independently
*Socially valid intervention
*Procedure: Child gets a pass at bedtime
Can use it 1 time out of bed
138
Procedures that are higher in social validity:
a. May result in better program adherence
b. Are typically less effective than other procedures
c. Should only be considered if initial procedures are ineffective
a. May result in better program adherence
139
Direct Techniques for the Social Validity of Procedures
* Purpose:
- Assess What the client prefers & the use of the procedures
* Assessments used:
- Participant preference & sustained use
* Schedule for Assessment:
- Before, During & after treatment
140
Assessing Participant Preference for Procedures
* Expose individuals to the treatment(s)
| * Provide a choice using a concurrent-chains arrangement
141
Concurrent-Chains Arrangement
* 2 or more chain schedules available simultaneously
* Application in assessing social validity of procedures:
- Initial Link = Selection response
- Terminal link = Treatment as determined by initial link selection
142
Expose individuals to Treatment
* Determine # & types of treatment
* Correlate each with a different stimulus (e.g. colored card)
* Run each treatment with stimulus present for predetermined time
* Measure bx during treatment
143
Provide Choice Using Concurrent-Chains Arrangement
* Provide all stimuli (e.g. colored cards)
* Client selects stimulus (Initial link)
* Enter room, run session with selected treatment (terminal link)
144
During a concurrent-chains schedule participants:
a. Must complete both schedules simultaneously
b. Are permitted to switch btw chains at any link
c. Are provided a choice btw stimuli previously correlated with different treatment conditions
c. Are provided a choice btw stimuli previously correlated with different treatment conditions
145
Assessing Social Validity of Procedures: Wilder et al. (2019)
* Identify maintaining consequence
* Expose to each TX: Break alone, edible, break with toys
* Provide opportunity to chose
146
Wilder et al. (2019): Implications
* Only shows what is preferred of the given options
* Provides empirical support for treatment
* May enhance the effectiveness of treatment
147
Assessing Maintenance/ Sustained Use of Procedures
* Observe after services have been delivered
* Determine if caregivers are still using procedures
- Sustained use of procedures may indicate high social validity of those procedures
148
When assessing direct preference for treatment, what conditions would provide the most complete assessment of social validity?
a. Multiple treatment conditions
b. One treatment condition & one no-treatment
c. Multiple treatments & a no-treatment condition
c. Multiple treatments & a no-treatment condition
149
Treatment Integrity as a Measure of Social Validity of Procedures
* Social validity of procedures asks these ques:
- Do the caregivers find the intervention feasible?
- Can the intervention be implemented with fidelity in the natural env?
* Assessing treatment integrity is one way to ans these questions
150
Treatment Integrity
*The extent to which the procedure is carried out as intended
151
Treatment integrity Vs. Procedural Fidelity
* Treatment integrity: Tx condition implemented as designed
| * Procedural fidelity: Adherence to all conditions of research design (BL & Tx)
152
Treatment integrity as a Measure of Reliability
* Dependent variable of interest is Bc
- Collect IOA to assess accuracy of measurement of DV
* Independent variable manipulated is treatment/ interventions
- Collect Tx integrity to assess accuracy of implementation of IV
153
Reasons for Tracking Treatment integrity
* Implementation of interventions can be subject to errors
* Accuracy of implementation is imp for:
- Client progress
- Program evaluation
* Good Tx integrity maybe an indication of high social validity
154
Problems with Low Treatment Integrity
* Clients may:
- Acquire new skills more slowly
- Continue engaging in prob bx for a longer period of time
* Implementers may:
- Abandon effective tx
- Use more restrictive Tx
155
IOA is a measure of accuracy of the ____.
DV
156
Treatment integrity is a measure of accuracy of the ____.
IV
157
List some possible results of poor treatment integrity:
Clients may acquire skills more slowly; implementers might abandon an effective treatment
Poor treatment integrity might result in a lack of desired results. Subsequently, implementers might abandon interventions that could have been effective if implemented accurately.
158
Steps to Assessing Treatment Integrity
1. Select which Tx to measure
2. Define the steps of the selected Tx
3. Determine the type pf Tx integrity data to use (global or specific)
4. Measure implementation through direct observation
5. Compare data collected to predetermined criteria
6. Train implementer if needed
159
Select Treatment to Measure
* Identify the precise procedure to be monitored
- E.g. of TX:
- Bx reduction procedures
- Mand training
- Discrete trials
- Natural Env Teaching
160
Define Steps of Selected Treatment
* Outline each step
* Create a data sheet to use
* Decide how many trials to observe
- Snapshot of Implementation Vs. each & every trial
161
When assessing treatment integrity:
a. Use the same data sheets across all procedures
b. Use the same measures for all implementers
c. Use data sheets specific to the client & treatment
c. Use data sheets specific to the client & treatment
162
Determine Type of Treatment Integrity Data to Use
*2 types of Tx integrity Data:
-2 Types of Tx integrity data:
Global scores
Score on specific components
163
*Global Score
*Reflects the total % of steps correct for the identified procedure
-When to Use:
Experienced implementer (few errors)
*Consideration:
-Hides info on components of Tx
164
Score on Specific Components
* Measure each specific step within a procedure
- Will be different for each Tc
* When to use:
- Inexperienced implementer
- Lowe global score for experienced implementer
165
Measure Implementation
* Observe & record implementer Bx
- Live (during TX) or review video recording of session
* Troubleshoot any issues with data collection system
* Graph data
166
Compare Data to Predetermined Criteria
* When analyzing the graph, look for:
- Trends (increasing or decreasing) in accurate implementation
- Effect on client bx
167
All treatment integrity scores must be conducted live.
a. True
b. False
b. False
| Treatment integrity scores can be conducted either live or by reviewing a video-recorded session.
168
When graphing treatment integrity data:
-Display with relevant client bx measures when appropriate
169
Omission Error
* Failing to implement a step as designed
| - E.g: Did not reinforce an appropriate request
170
Commission Error
* Applying intervention when not indicated by plan
| - e.g: Gave Rx for inappropriate response
171
Treatment Integrity Errors in the Literature E.g. 1
* Examined omission Vs. Commission errors during a differential Rx procedure
* Reinforcing prob Bx (Commission error) was more problematic than failing to Rx the replacement (Omission error)
* Order of conditions also impacted results
172
Treatment Integrity Errors: General Conclusions from Research
* Commission errors are more detrimental than omission errors
- E.g. providing a rx when it should not be delivered is worse than not providing it when it should have been delivered
* Sequence effects have an impact
- E.g. full integrity in BL to lower integrity during TX
173
Select all the omission errors:
a. Did not deliver Rx as planned
b. Issued a prompt at the wrong time
c. Delivered a RX after an incorrect response
d. Did not prompt as designed
a. Did not deliver Rx as planned
| d. Did not prompt as designed
174
The studies by St. Peter Pipkin et al., and Leon et al., both showed ____ errors to be more detrimental to both acquisition and behavior reduction.
Commission
Both these studies showed commission errors were more detrimental in both acquisition and behavior reduction programs. While omission errors might make a procedure have no effects, commission errors can make a procedure have unintended effects.
175
Reasons for Low Treatment Integrity
- Complex protocol
- Lack of Generalization
- Tx "drift"
- Competing contingencies
- Lack of training
176
Complex Protocol
* When required to record several types of data or complete several steps simultaneously
* Evident when plan is being executed, but not fully
- E.g. error in prompt level while managing distracting materials
* Solution: Clear descriptions, simplify
177
Lack of Generalization
* Not executing the plan in new Setting or with similar Bx
* Evident when there are omission errors or cannot Tact the expectation
e. g. Prompt child to respond to specific peer, but miss the natural occasion
* Solution: Train across multiple settings/situations
178
Treatment "Drift"
* Tx applied differently over time
* Evident when it is applied correctly at times but varies with intensity or "correctnesS"
- E.g. consequating volumes of yelling
* Solution: Careful monitoring, booster training sessions
179
Competing Contingencies
* Rx for departing from protocol or punishment for adherence
* Evident when engaged in some other task that is not program-related
e. g. cooking dinner instead if attending to appropriate Bx
* Solution: Prioritize, Training, Feedback
180
Lack of Training
* Inadequate/ Incomplete training
* Evident when individual can't describe what to do & has never been observed doing it correctly
- E.g. prompts mand for toy, delivered praise
* Solution: Competency-based training across exemplars & contexts
181
Train Implementer
* Not meeting criteria
| - May have to train specific component or entire procedure
182
Uses of Treatment Integrity Data
* Feedback
- Immediate
- Cumulative
* Clinical decision-making
* Supports IOA
* Further Data
* Procedural requirement
183
For the past year, Nikita has been carrying out a reduction procedure to reduce the intensity of head slaps. Lately intensity has increased, but reduction procedures have not been used. What is the likely issue?
a. Lack of generalization
b. Treatment drift
c. Competing contingencies
b. Treatment drift
184
For the past year, Nikita has been carrying out a reduction procedure to reduce the intensity of head slaps. Lately intensity has increased, but reduction procedures have not been used. What is the suggested solution?
a. Booster training sessions
b. Reprioritize tasks
c. Simplify procedures
a. Booster training sessions
185
Formal Reassessment
* Periodic comprehensive assessment to identify progress toward goal, or to reevaluate existing prob bx
- Resembles initial assessments, but typically brief
- E.g: Curricular assessments, Functional Bx assessments
186
Reasons for Conducting Formal Reassessments
- Client progress
- Required by funding sources
- Organizational policy
- Curriculum guidelines
187
Formal Reassessment Due to Client Progress
* Most imp reason for formal reassessment
* Client progresses quickly
- E.g: Previously had no mands, now using 3-word sentences
* New bx develops
188
Formal Reassessment Due to Funding Source
* Funder (e.g., insurance company) may authorize Tx for a specific period of time & reauthorization may require a formal assessment
- e.g: Updated assessment scores
* Often time allowed for reassessment less than for initial assessment
189
Formal Reassessment Due to Organizational Policy
*Companies may require formal reassessments on a specific schedule
-E.g.
End of semester/ term
Parent reports (annual; semi-annual)
190
Formal reassessments conducted after treatment has been in place should be:
a. More thorough & longer than initial assessments
b. Similar, but take less time
b. Similar, but take less time
191
Social Validity of Outcomes/Results
*Acceptability of the effects of the procedures
| "Are consumers satisfied with the results? All the results, including any unpredicted ones?"
192
Intended & unintended Outcomes/ Results
* Intended: Results as initially targeted
* Unintended: Result that was not targeted
* Accountable for both!
* Anticipate the side effects as best as possible
- Risk-benefit analysis
193
*Unintended Results:
-Must still be assessed
194
3 levels of Outcomes/ Results
* Assess social validity at these levels:
- Proximal (Short-term)
- Intermediate
- Distal (Long-term)
195
Assessing the immediate results of treatment is what level of results assessment?
a. Distal
b. Intermediate
c. Proximal
c. Proximal
196
The different levels of results assessment align with different:
a. Procedures
b. Points in time
c. Locations
b. Points in time
197
Methods for Assessing the Social Validity of Outcomes/ Results
* Indirect techniques
- Interviews
- Questionnaires
- Rating scales
* Direct techniques
- Expert evaluation
- Normative comparison
- Norm-referenced (standardized) assessment
- Test in Natural Env (real-world test)
198
Indirect Techniques for the Social Validity of Outcomes/ Results
* Purpose:
- Assess stakeholder opinion about results of services
* Assessments used:
- Interviews, Questionnaires, Rating scales
* Schedule for assessment
- Post-intervention
199
Direct Techniques for the Social Validity of Outcomes/ Results
* Purpose:
- Evaluate bx of client compared to criteria for Bx change
* Assessments used:
- Expert evaluation, normative evaluation, standardized tests, test in the natural env(real-world test)
* Schedule for assessment:
- Post-intervention
200
Expert Evaluation for Assessing Social Validity of Results
*Outside observer reviews a recorded pre-intervention performance & compares to post-intervention performance
-E.g.
Training employee to greet customers, managers views several clips and rates each
201
Normative Comparison for Assessing Social Validity of Results
* Identify typical performance & compare dimension of client performance to typical performance
* E.g:
- Classroom Bx: In seat, attending, taking notes, responding to ques
202
Normative Comparison for Assessing Social Validity of Results
*Use a Norm-referenced Assessment (Standardized test) & compare client score to typical population
E.g.
-IQ scores
-Academic achievement tests
203
Test in Natural Environment for Assessing Social Validity of Results
* Test to see if the Client's change of Bx sustains over a period of time & into other context(s)
* E.g:
- Om the job
204
Test in Natural Environment for Assessing Social Validity of Results
* Test to see if the Client's change of Bx sustains over a period of time & into other context(s)
* E.g:
- On the job
- On the street
- In a restaurant
205
What do expert evaluation and consumer evaluation have in common?
a. Raters should not know if the performance is pre- or post- intervention
b. Both are conducted during live performance
c. Each rater will design their own scale
a. Raters should not know if the performance is pre- or post- intervention
206
Results of IQ tests:
a. Could be used to evaluate results if valued by consumer
b. Have no bearing on social validity of results
c. Are standard practice to use in bx analysis
a. Could be used to evaluate results if valued by consumer
207
Generality Dimension of ABA
*"A behavioral change maybe said to have generality if it proves durable over time, if it appears in a wide variety of possible env, or if it spreads to a wide variety of related bxs"
208
Social Validity & the Generality Dimension
* When the Bx change becomes part of Natural setting
| * Lasts over time (maintenance of Bx change)
209
Effective Dimension of ABA
*It's (application of behavioral technique) practical importance, specifically its power in altering Bx enough to be Socially imp, is the essential Criterion"
210
How does generality relate to social validity?
Social validity will measure if the results of intervention are seen in the appropriate setting and are sufficiently long-lasting
Social validity is related to the importance of interventions to consumers. An intervention that is important for consumers is one that produces results in the natural env with those results persisting over time.
211
How does the effective dimension relate to social validity?
Social validity will measure the extent to which consumers find treatment effective
Social validity is related to the importance of interventions to consumers. An intervention that is important for consumers is one that produces results that are big enough to have a meaningful impact on their life.
212
After Outcomes Have Been Met
* Terminate services once client has reached the predetermined outcomes
- Terminate services: Discontinue the professional relationship
* This is the reason that Objective outcomes have been defined prior tp beginning service delivery
213
Criteria for Terminating Services
* Services are not Needed (cuz outcomes have been met)
* Client is not benefitting from services
* Client is being harmed by services
* Env is unsafe
214
Before Terminating Services
* Plan ahead with client & caregivers
* Collaborate with other professionals
* Present a comprehensive report for how services will be terminated
- What type of support still needed?
- Ongoing
- Monitoring
215
When Terminating Services
* Be sure to adequately prepare for transition
* Discuss with all interested parties
- Client & other stakeholders
* Bx analysis never abandon clients
216
Social Validity: Take-Home Points
* We are accountable
- Improve lives without imposing our beliefs on others
* Assessment is ongoing
- Objectively measure bx & subjectively evaluate the value of the goals, procedures and results
217
Select the relevant reasons to terminate services:
a. Outcomes have been met
b. Social validity appears low
c. Parties do not agree with initial goals
d. Environment is unsafe
a. Outcomes have been met
| d. Environment is unsafe
218
Once goals are met, transitions should be:
a. Swift
b. Planned for and arranged with relevant parties
c. Conducted only when funding expires
b. Planned for and arranged with relevant parties
5047
flashcards
Decks in class (9)
# Cards
