1
Q
Why must the order of variables in hierarchical regression be justified?
A
Because the order determines which variables control for variance before others are tested
2
Q
What does commonality analysis measure?
A
The unique and shared contributions of predictors to the variance explained in the dependent variable.
3
Q
What concept determines the order of variables in hierarchical regression?
A
Causal priority
4
Q
Why should demographic variables often be entered first in hierarchical regression?
A
Because they are usually considered control variables that precede psychological or behavioral predictors.
5
Q
What follow-up technique did Petrocelli recommend for understanding predictor contributions?
A
Commonality analysis
6
Q
What is multicollinearity?
A
A condition where predictor variables are highly correlated with each other
7
Q
Why is ΔR² important in hierarchical regression?
A
It shows the unique contribution of a new set of predictors beyond those already in the model
8
Q
What mistake occurs when researchers try to maximize R² in hierarchical regression?
A
They treat hierarchical regression like stepwise regression rather than theory-driven analysis
9
Q
What statistics are commonly used to detect multicollinearity?
A
Variance Inflation Factor (VIF) and tolerance.
10
Q
Why must the order of variables in hierarchical regression be justified?
A
Because the order determines which variables control for variance before others are tested
11
Q
When does measurement error typically attenuate regression coefficients?
A
When measurement errors are uncorrelated with each other
12
Q
Why is normality less critical in large samples?
A
Because the Central Limit Theorem makes statistical estimates approximately normally distributed.
13
Q
What are the two misconceptions about multiple regression addressed in the article?
A
Variables must be normally distributed.
Measurement error always attenuates regression coefficients
14
Q
What happens when homoscedasticity is violated?
A
Heteroscedasticity occurs, meaning residual variance changes across predictor values.
15
Q
Why is normality of residuals important in regression analysis?
A
It helps ensure accurate hypothesis tests and confidence intervals, especially in small samples
16
Q
Does multiple regression require predictor variables to be normally distributed?
A
No, predictors do not need to be normally distributed.
17
Q
What happens when measurement errors are correlated?
A
Regression coefficients may be biased either upward or downward
18
Q
What is effect size?
A
A quantitative measure of the magnitude of a relationship or difference between variables.
19
Q
What effect size measure is commonly used for ANOVA?
A
f
20
Q
What is sensitivity analysis in power analysis?
A
Determining the smallest effect size that can be detected with a given sample size, α, and power
21
Q
How does effect size influence statistical power?
A
Larger effect sizes increase statistical power
22
Q
What is criterion power analysis?
A
Determining the significance level (α) needed to achieve a desired level of power with a fixed sample size and effect size
23
Q
What effect size measure is commonly used in multiple regression?
A
𝑓^2
24
Q
What is the formula for statistical power?
A
Power = 1−𝛽
25
What happens if a study is underpowered?
It may fail to detect real effects, increasing the likelihood of Type II errors
26
What level of statistical power is commonly recommended in research?
0.80 (80% probability of detecting a true effect).
27
What does the regression rule 𝑁≥104+𝑚 help researchers evaluate?
The significance of individual predictors
28
What is a common sample size guideline for experimental studies?
Approximately 20–30 participants per group.
29
What are the four main factors that influence statistical power?
Effect size
Sample size
Significance level (α)
Variability in the data
30
What is a “rule of thumb” in sample size determination?
A general guideline used to estimate sample size when formal power analysis is unavailable
31
What is a commonly suggested minimum sample size for correlation studies?
Around 30 participants
32
What rule of thumb did Green (1991) propose for testing individual predictors in regression?
N≥104+m
33
Why should rules of thumb be used cautiously in research?
They do not account for effect size, study design, or variability
34
What is the recommended process for determining sample size?
Estimate effect size, set α, choose desired power, and conduct a power analysis
35
What is a partial correlation?
The relationship between Y and a predictor after removing the effects of other predictors from both variables
36
What are orthogonal predictors?
Independent variables that are not correlated with each other
37
One solution to multicollinearity problems?
Remove redundant predictors or use ridge regression
38
What does DFFITS measure?
The influence of a data point on its own predicted value
39
What does independence of errors mean?
Residuals are not correlated with one another
40
Difference between partial and semi-partial correlation?
Semi-partial removes shared variance from the predictor only, while partial removes shared variance from both the predictor and the outcome
41
What is an influential observation?
A data point that significantly changes the regression coefficients or model fit
42
What does DFBETAS measure?
The influence of an observation on individual regression coefficients
43
What is a semi-partial (part) correlation?
The relationship between Y and a predictor after removing the effect of other predictors from that predictor only.
44
What does Cook’s Distance measure?
The overall influence of an observation on the regression model.
45
What important interpretation comes from squared semi-partial correlation?
It equals the change in R² when that predictor is added to the model.
46
What are studentized residuals (internal)?
Residuals scaled by their estimated standard deviation that accounts for leverage
47
What does R² represent in multiple regression?
The proportion of variance in the dependent variable explained by the entire regression model.
48
Which residual type is best for detecting influential outliers?
R-student (external studentized) residuals
49
Why is R² not sufficient for understanding predictors?
Because it shows total model fit but does not show the unique contribution of each predictor
50
Why are correlated predictors more complicated in regression?
Because predictors share overlapping variance, making it difficult to determine each predictor’s unique contribution
51
can a model have a large R² but no significant predictors?
yes, when predictors are highly correlated and share overlapping variance.
52
What is the primary goal of multiple regression?
To determine how much variance in the dependent variable (Y) is explained by a set of independent variables (predictors)
53
What is the QQ plot used for?
To assess normality of residuals, focusing on deviations in the tails
54
Tests for constant variance (homoscedasticity) include__
Breusch-Pagan test, Bartlett’s test, Levene’s test.
55
When plotting residuals vs fitted values, what indicates a problem?
Patterns such as funnels (heteroscedasticity) or curves (nonlinearity); a random scatter indicates assumptions are met
56
Purpose of residual plots?
To check linearity, constant variance (homoscedasticity), normality of residuals, and identify outliers or influential points
57
What complication arises when IVs are correlated?
The individual R²s for IVs do not sum to the overall model R², making it harder to determine unique contributions
58
Define semi-partial (part) correlation
The correlation between Y and an IV after removing the variance explained by other IVs from that IV only
59
Metrics to assess influential observations include__
Cook’s Distance, DFFITS, DFBETAS
60
Name four methods for scaling residuals.
Standardized residuals, studentized (internal) residuals, PRESS residuals, R-student (external) residuals
61
When should you report semi-partial correlations?
When you are interested in the unique contribution of a given independent variable to Y.
62
Why is normality of residuals important?
Needed for t-tests, F-tests, and confidence intervals; estimates of coefficients can still be obtained if normality is violated, but inference may be invalid.
63
What is Weighted Least Squares (WLS)?
A regression method where each observation is weighted inversely to its error variance to handle heteroscedasticity
64
What is the purpose of centering variables in regression?
Improves interpretability (especially intercept), reduces multicollinearity for polynomials and interactions, without changing model fit.
65
Why are transformations used in linear regression?
To achieve linearity, stabilize residual variance (homoscedasticity), and improve normality of residuals.
66
What is the purpose of linearizing transformations?
To make the relationship between X and Y linear, meeting regression assumptions
67
What issues arise with polynomial models?
Overfitting, extrapolation errors, multicollinearity, interpretability of coefficients.
68
What are the two main types of transformations?
1. Linearizing transformations – usually on predictors (X)
2. Variance-stabilizing transformations – usually on response (Y)
69
Name some common transformations to linearize data.
Logarithm, square root, power, reciprocal, exponential.
70
When is using Box‑Cox particularly recommended?
When skewness is severe
-When traditional transformations do not sufficiently normalize the data
-When the relationship between variance and mean is non-linear
71
What is the effect of Box‑Cox transformations on correlation and regression coefficients?
By improving normality and stabilizing variance, it can increase the reliability and interpretability of correlations and regression results
72
How do modern statistical packages help with Box‑Cox transformations?
They can automatically estimate the optimal λ and provide visualizations or likelihood plots to guide the transformation
73
Can Box‑Cox be applied to skewed data in both directions?
Yes, it can handle positively or negatively skewed data, often by adjusting the data beforehand
74
Why must predictions be back-transformed?
To interpret results in the original units of Y.
75
What is the Box-Cox transformation when λ = 0?
Y^(0)=log(Y)
76
What is the Box-Cox transformation formula (λ ≠ 0)?
Y^(λ)=Yλ−1/λ
77
What transformation corresponds to λ = 1?
No transformation (approximately the original data)
78
What is the purpose of the Box-Cox transformation?
To improve linear regression assumptions by reducing non-normality, non-linearity, and heteroscedasticity.
79
What is the inverse (back-transformation) formula for λ ≠ 0?
Y=(λ⋅Y^(λ)+1)^1/λ
80
In a log(Y) ~ X model, how is β₁ interpreted?
As an approximate percentage change in Y for a 1-unit increase in X
81
What does a model Y ~ log(X) represent?
Change in Y for a percentage change in X.
82
Why is interpretation in original units important?
It makes results understandable and meaningful in real-world terms.
83
What happens to confidence intervals after transformation?
They apply to the transformed scale and must be interpreted carefully.
84
What should researchers always report when using transformations?
The type of transformation and the reason for using it.
85
What does an S-shaped curve in a Q–Q plot indicate?
Skewness in the data.
86
What does each point in a Q–Q plot represent?
A pair: (theoretical quantile, sample quantile).
87
What are the axes of a P–P plot?
Theoretical probabilities vs. empirical probabilities.
88
Which plot is more sensitive to differences in the center of the distribution?
P–P plot
89
What are the axes of a Q–Q plot?
Theoretical quantiles vs. sample quantiles.
90
What indicates a good fit in a Q–Q plot?
Points lying close to a straight 45° line
91
When is a P–P plot commonly used?
For checking overall distribution fit.
92
What is the rule of thumb for DFFITS?
|DFFITS| > 2√(p/n)
93
What is a leverage point in regression?
A point with an unusual X-value (far from the mean of predictors).
94
What does COVRATIO > 1 indicate?
The point improves precision.
95
What is an alternative to removing influential points?
Use robust regression methods.
96
What does COVRATIO ≈ 1 indicate?
The point has little effect on model precision.
97
What is the main purpose of regression diagnostics?
To identify points that distort model fit and estimates.
98
What two factors typically make a point influential?
High leverage and a large residual.
99
What does Cook’s Distance combine?
Residual size and leverage.
100
What does DFBETAS measure?
The influence of a point on each regression coefficient β̂ⱼ.
101
What does a large Mahalanobis distance indicate?
A potential multivariate outlier.
102
Does high leverage always mean a point is influential?
No, only if it also has a large residual.
103
What is the rule of thumb for high leverage?
hᵢᵢ > 2p/n
104
When should you remove an observation?
If it is an error, invalid, or not part of the population.
105
Why compare models with and without influential points?
To assess their impact on results.
106
What does DFFITS measure?
The influence of a point on its own predicted value.
107
What is the rule of thumb for DFBETAS?
|DFBETAS| > 2/√n
108
What are standardized residuals?
Residuals converted into z-scores using a common variance estimate.
109
What are common causes of outliers?
Data entry errors, measurement errors, or true extreme values.
110
Does high leverage guarantee influence?
No
111
What distribution do studentized residuals follow?
Approximately a t-distribution.
112
What should you do after identifying influential points?
Re-run the model with and without them.
113
Why are studentized residuals preferred over standardized residuals?
They are more accurate because they account for unequal variance.
114
What is the key question diagnostics help answer?
Is the model being overly influenced by a few unusual observations?
115
What is a common cutoff for Cook’s Distance?
> 4/n or > 1
116
What is an outlier in regression?
A case with an extreme value in the response variable (Y)
117
What is the range of leverage values?
0 to 1
118
What is an influential observation?
A point that significantly changes regression results when removed.
119
What does VIF > 5 indicate?
High collinearity (investigate)
120
What does p ≤ 0.05 mean for normality tests?
Residuals deviate from normality.
121
What does VIF > 10 indicate?
Very high collinearity (problematic).
122
What does VIF between 2–5 indicate?
Moderate collinearity (monitor).
123
What kurtosis range is acceptable?
±3
124
What does Pr(>F) represent?
p-value for significance of predictors.
125
What does Sum of Squares (SS) represent?
Variation explained by predictors.
126
What is semi-partial (part) correlation?
Unique contribution of a predictor to the outcome.
127
When is leverage considered high?
When hᵢᵢ > 2p/n
128
What does part correlation indicate?
Unique variance explained by that predictor.
129
What does zero-order correlation show?
Raw, uncontrolled relationships.
130
What is mesokurtic distribution?
Normal-shaped (baseline).
131
What does the F-value represent?
Ratio of model variance to residual variance.
132
What does VIF between 1–2 indicate?
Low collinearity (acceptable)
133
What is platykurtic distribution?
Flatter with light tails (fewer outliers).
