What is a process model?
A process model represents an overview of the explanatory variables and the associations between the variables and the outcome variable
Why do we need a process model?
A process model provides a structured, graphic overview of relevant variables and their relationships to each other and the outcome variable.
How do we build a process model?
There is a set of heuristics and guidelines to use that guide us through the process of building a process model that fits our problem definition and analysis.
What are the different types of associations?
- Direct
- Indirect
- Interaction
- Reinforcing
- Undermining
Direct association
A causes B
A => B
Indirect assocation
- A causes B through M
- Also called ‘mediation’
- Many behavioural psychological associations are in some shape or form indirect.
A => M => B
Interaction association
- A causes B
- S influences the strength of association between A and B
- Also called ‘moderation’
A => B
S => (A => B)
Reinforcing association
- A form of an interaction
- X strengthens the association between A and B
- The relationship between A and B is stronger when X has a higher value
A => B
X => (A => B)
Undermining association
- A form of an interaction
- Y weakens the association between A and B
- The relationship between A and B is weaker when X has a higher value
Y => (A => B)
What are the heuristics for building the process model?
- Make a list of the explanations/variables
o This is the end point of your analysis phase - Check that all the variables are psychological, specific, concrete, and continuous
- Check that all the variables are related to behaviour, attitudes, or affect
- Start by putting the outcome variable on the far right side of the model
- Work from right to left
o Ask yourself: which variables affect the outcome variable?
o On the left are the explanatory variables - Draw arrows between the variables in your model to depict the direction of effects using + and –
- Make sure that the associations between your variables and outcome variables are not too remote
- Are the associations direct, indirect, or interactions?
o You do this mostly based on the literature from the analysis phase - Consider whether variables are also related to each other
- Keep it operational: +/- 10 variables (excl. outcome variable)
- Keep it operational: +/- 4 steps between outcome and far left of the model
o You don’t take more than 4 steps, because the further away your variable is from the outcome variable, the more diluted, and thus weaker, the association becomes. This makes it less interesting to focus on.
The heuristics of creativity in employees.
A list of factors that affect the creativity in employees based on the analysis phase:
- Stress
- Mood
- Cognitive resources
- Time pressure
- Technical skills
- Number of tasks
- Competition
- Self-efficacy
- Etc.
- You start by putting your outcome variable on the far right
o In this case: creativity score - Then you put the variables that are most directly related to your outcome variable, together with their direction (+/-)
o In this case: cognitive resources, mood and mental fatigue - Then you start working further to the left
o In this case: stress
o Further to the left in this case: competition, time pressure, work pressure, self-efficacy on stress - If a variable has an indirect effect on the outcome variable, this doesn’t mean that it can’t also have a direct effect, so you should look at this.
- Finaly you look at possible moderators
Why do you have to select variables
The list of variables from the analysis is longer than the list of variables that you want to include in the model, in order to keep it operational and workable.
How do you select variables?
Psychological
- Is it a psychological variable?
- If it’s not, it automatically becomes less interesting for a psychological process model.
Modifiability
- Can the variable be changed/influenced?
- If a variable is very static it might not be that interesting because you’re looking for variables to target with your intervention.
Strength of association
- Is the association strong enough to be interesting?
- It should not be too weak or too remote.
Context
- Is the context of evidence transferable?
- Is the context of the evidence that you found still applicable to your case?
Evidence base
- Is there enough high level evidence for the association to be supported in your model?
What does testing the model mean?
Testing the model means that you zoom in on all the different arrows and you need to be able to support all these arrows by the evidence that you have.
The evidence that you have can have different levels of evidence base. What are these levels of evidence?
Opinions:
- The sixth level; the lowest level
- It might be from experts but they’re only opinions
Case studies:
- The fifth level
- Some level of evidence but not so much, especially if you only have one or two
Observational studies, pre-post measurements:
- The fourth level
- Some level of evidence but not so much, especially if you only have one or two
Non-randomized controlled trial
- The third level
Randomized controlled trials
- The second level
Systematic reviews and meta-analysis
- The first level; the highest level
- They support the association that you have represented in your model
- These are always good to look at because they tend to summarize the evidence that is out there.
- They give you information about the strength of the association
- They often give information about the evidence base that underlies the review or meta-analysis and how strong and what level that is
P-values vs effect sizes
We want to look at the actual statistical effect sizes or the strength of association that underlies each of the arrows in our model.
Go beyond the p-value and look at the effect sizes
- The p-value tells us something, but not so much about the strength of association.
Statistical significance testing cannot evaluate result importance. Cohen (2994) observed that the statistical significance test ‘does not tell us what we want to know, and we so much want to know what we want to know that, out of desperation, we nevertheless believe that it does!’
So, p-value tells us something about the statistical significance, which is important, but we are more interested in the strength of association.
Effect size
A statistic that indicates to what extent sample results differ from the expectations as phrased in the null hypothesis (i.e., no difference between groups, no association between variables).
I does not tell us if it’s statistical significance but how strong the relation is. They are often related to each other.
Effect size (ES) can be in original measurement units (e.g., ms, scores on a scale) or standardized (e.g., Cohen’s d, beta, eta, r).
Is the association/effect size based on a large quantity of studies (e.g., a meta-analysis) or a single or small set of studies?
- One effect size is no effect size
- The more you have to support the effect size that you are reporting, the stronger your evidence base is
What do you need to know about the number of people when looking at effect sizes?
How large is N?
- How many people are included in the different studies?
Who is N?
- Who are the participants?
- This tells you something about how generalizable the effects are
Tips and tricks of the process model
Find psychological mediators and moderators.
Are all variables concrete?
- Not ‘attitudes’ but ‘attitudes towards stair use’
- Not ‘demands’, but ‘experienced time pressure’
Stick to one perspective in your model
- E.g.., when researching ‘bullying’ focus on the bully, the victim, or the bystanders, but not all of them.
Start with the problem: what needs to change?
Support all associations in the model.
Be complete, but keep it workable.
What is the end product of the test phase?
A model that represents the different associations between variables from the analysis phase and the outcome variable.
