Module 8 - Regression Analysis

Question 1

What is the definition of regression analysis?

Answer 1

Describes a statistical relationship between variables

The dependent variable responds to the independent variable(s) with measurable uncertainty.

Question 2

What is the equation for the most basic regression model?

Answer 2

y = a + bx + ϵ

Here, a is the y-intercept, b is the slope, and ϵ is the error term.

Question 3

What does homoscedasticity refer to in regression analysis?

Answer 3

Constant variance of normal errors independent of X

This property is essential for valid regression results.

Question 4

What are the two crucial results of regression analysis for CER development?

Answer 4

• Establishing statistical significance of CERs
• Quantifying uncertainty in a risk model

These results help in making reliable cost estimates.

Question 5

What does Ordinary Least Squares (OLS) regression aim to minimize?

Answer 5

Sum of Squared Errors (SSE)

OLS regression is a foundational method for determining the best-fit line.

Question 6

What does R² indicate in regression analysis?

Answer 6

Percent of overall variation in cost explained by the regression equation

The complement, 1 − R², indicates unexplained variation.

Question 7

What is the Standard Error of the Estimate (SEE) used for?

Answer 7

Measures accuracy of predictions made by the regression line

It estimates the standard deviation of the normally-distributed error term.

Question 8

What does the F-test determine in regression analysis?

Answer 8

Whether the regression model is statistically significant

This test assesses the overall fit of the model.

Question 9

What is the purpose of the t-test in regression analysis?

Answer 9

Determines whether individual cost driver variables are statistically significant

It helps in validating the relevance of each predictor.

Question 10

What is a proxy variable in cost estimating?

Answer 10

An independent variable that stands in for one which drives cost

Example: Number of firemen as a proxy for fire size.

Question 11

What is the role of scatter plots in regression analysis?

Answer 11

Visualize patterns and detect correlation between variables

They help in determining the appropriate regression model.

Question 12

What does the correlation coefficient (ρ) indicate?

Answer 12

Strength and direction of the relationship between variables

Ranges from -1 (perfect negative) to +1 (perfect positive).

Question 13

What is the difference between correlation and causation?

Answer 13

Correlation indicates a relationship; causation indicates one variable drives another

Causation cannot be statistically verified.

Question 14

What is the first step in regression analysis?

Answer 14

Create a scatter plot of the data

This helps to visualize correlation and determine the model type.

Question 15

What are the two primary practical applications of regression in cost estimating?

Answer 15

• CER development
• Learning curve analysis

These applications leverage historical data for future cost predictions.

Question 16

What is the purpose of confidence intervals (CIs) in regression?

Answer 16

Quantify uncertainty regarding the estimate of mean cost

They provide error bounds for the mean of the estimate.

Question 17

What does multicollinearity refer to in OLS multivariate regression?

Answer 17

Risk introduced by multiple independent variables

It can distort the regression results.

Question 18

What is the general guideline when using statistical software for regression?

Answer 18

Always plot the data and fit a trend line first

This helps guide and check the regression analysis.

Question 19

What is the best estimate for any value of the cost driver variable?

Answer 19

Result of plugging that x-value into the best-fit regression equation

This provides the predicted cost based on the model.

Question 20

What is a non-OLS model?

Answer 20

Models like Weighted Least Squares (WLS) and Mean Absolute Percentage Error (MAPE)

These models minimize errors other than SSE.

Question 21

What is the learning curve analysis in cost estimating?

Answer 21

Application of regression to understand cost reductions over time with increased production

It helps in forecasting future costs based on experience.

Question 22

What is the first step of regression analysis?

Answer 22

Making a scatter plot

A scatter plot provides insight into the correlation’s strength and helps determine the best model type.

Question 23

List the basics of linear regression analysis.

Answer 23

• Finding the regression equation
• Determining the goodness of fit
• Calculating confidence intervals

These basics are essential for understanding the model’s performance.

Question 24

The key concept of regression is to use the method of least squares for determining what?

Answer 24

The best estimates for equation parameters

This method minimizes the sum of the squared differences between the data points and the estimated line.

Question 25

What does **SSE** stand for in regression analysis?

Answer 25

Sum of Squared Errors ## Footnote SSE provides a metric for the total amount of uncertainty surrounding the regression line.

Question 26

True or false: In OLS, residuals are squared before addition to avoid cancellation.

Answer 26

TRUE ## Footnote Residuals can have both positive and negative values, so squaring them ensures they contribute positively to the sum.

Question 27

What does the **y-intercept** represent in a regression equation?

Answer 27

The point where the line passes through the mean of the X and Y data ## Footnote It is calculated using the formula: ( hat{a} = ar{Y} - hat{b} ar{X} ).

Question 28

What does the **error term** ( epsilon ) describe in regression analysis?

Answer 28

Statistical uncertainty surrounding the equation ## Footnote It represents the residuals and indicates that the relationship is statistical rather than purely functional.

Question 29

List the **two key OLS assumptions** regarding the error term.

Answer 29

* It is independent with X * It has a probability distribution ## Footnote These assumptions are critical for the validity of the regression model.

Question 30

What is the purpose of calculating **confidence intervals** in regression analysis?

Answer 30

To estimate the range within which the true parameter values lie ## Footnote Confidence intervals provide insight into the reliability of the predictions made by the model.

Question 31

What is an **irreducible component** in the context of data analysis?

Answer 31

An error term that will always persist despite improvements in data collection and normalization ## Footnote It indicates that some level of error is inherent in the data.

Question 32

What are the two key **OLS assumptions** regarding the error term?

Answer 32

* Independence with respect to X (homoscedasticity) * Normally distributed with a mean of zero and constant variance ## Footnote These assumptions are crucial for the validity of Ordinary Least Squares regression.

Question 33

What does **homoscedasticity** refer to in OLS regression?

Answer 33

The assumption that the error term is independent and shows no correlation with X ## Footnote This means that the spread of the residuals should be constant across all levels of the independent variable.

Question 34

What is the purpose of **residual analysis** in OLS regression?

Answer 34

To check the OLS assumptions by examining a residual plot ## Footnote It helps determine if the regression model is appropriate for the data.

Question 35

What does the regression equation consist of in OLS regression?

Answer 35

* Fitted slope (b hat) * y-intercept (a hat) ## Footnote These coefficients are derived from minimizing the sum of squared errors (SSE).

Question 36

True or false: The error term in regression makes it more than just **curve fitting**.

Answer 36

TRUE ## Footnote The error term accounts for the variability in the data that cannot be explained by the model.

Question 37

What should a residual plot ideally portray in terms of data distribution?

Answer 37

Symmetry about the x-axis with constant variability ## Footnote This indicates that a normal distribution with a mean of 0 is a reasonable assumption.

Question 38

What are the three basic **measures of variation** calculated in ANOVA?

Answer 38

* Sum of Squares Total (SST) * Sum of Squared Errors (SSE) * Sum of Squares Regression (SSR) ## Footnote These measures help assess the goodness of fit of the regression model.

Question 39

The relationship between SST, SSE, and SSR can be expressed as __________.

Answer 39

SST = SSE + SSR ## Footnote This equation shows how total variation is partitioned into explained and unexplained variation.

Question 40

What does **SST** represent in ANOVA?

Answer 40

Total variation in Y ## Footnote It is calculated as the sum of the squared differences between each data point and the mean of the y-values.

Question 41

What does **SSE** represent in ANOVA?

Answer 41

Unexplained variation around the regression line ## Footnote It is calculated as the sum of the squared differences between each observed y-value and its corresponding predicted y-value.

Question 42

What does **SSR** represent in ANOVA?

Answer 42

Explained variation from the average to the regression line ## Footnote It is calculated as the sum of the squared differences between predicted y-values and the mean of the y-values.

Question 43

What is the significance of **degrees of freedom** in the context of SST, SSE, and SSR?

Answer 43

* SST: n-1 d.f. * SSR: k-1 d.f. * SSE: n-k d.f. ## Footnote Degrees of freedom are crucial for determining the statistical significance of the regression model.

Question 44

What is the **penalty** imposed when introducing new parameters in regression analysis?

Answer 44

Larger average variation due to loss of degrees of freedom ## Footnote This reflects the trade-off between model complexity and explanatory power.

Question 45

What is the formula for **Mean Squared Error (MSE)**?

Answer 45

MSE = SSE / (n - k) ## Footnote Where SSE is the sum of squared errors, n is the number of data points, and k is the number of parameters.

Question 46

What does **degrees of freedom (d.f.)** represent in regression analysis?

Answer 46

The value of the denominator for each mean measure of variation ## Footnote It imposes a penalty for each new parameter introduced in the model.

Question 47

In the context of regression, what does **Standard Error (s)** quantify?

Answer 47

Uncertainty in the estimated regression coefficients and the regression equation ## Footnote It indicates how much variability exists in the estimated values.

Question 48

What distribution do the standard errors of estimated coefficients follow?

Answer 48

t-distribution ## Footnote This occurs when estimating the mean of a normally distributed population with a small sample size and unknown population standard deviation.

Question 49

What is the **Standard Error of Estimate (SEE)**?

Answer 49

The square root of the Mean Squared Error (MSE) ## Footnote It provides a quantitative measure of the variability of an estimate or forecast made using the regression equation.

Question 50

How is the **Coefficient of Variation (CV)** calculated?

Answer 50

CV = SEE / mean of y-values ## Footnote It expresses the SEE as a percentage of the mean and indicates the percent error associated with an estimate.

Question 51

What significance level is commonly used in regression analysis?

Answer 51

0.05 (5%) ## Footnote This means the assumption of a significant cost driver when there isn't one only happens 5% of the time.

Question 52

What are the null and alternative hypotheses in a **t-test** for regression coefficients?

Answer 52

H0: b = 0; H1: b ≠ 0 ## Footnote The null hypothesis states that there is no relationship, while the alternative suggests there is a significant relationship.

Question 53

What is the formula for calculating the **t statistic**?

Answer 53

t = Estimated Coefficient / Standard Error ## Footnote This statistic is used to determine the statistical significance of the regression coefficients.

Question 54

What does a **p-value** indicate in regression analysis?

Answer 54

The probability that the correlation between x and y at its observed strength would arise if H0 is true ## Footnote A small p-value (less than significance level) indicates statistical significance.

Question 55

What is the **probability** that the correlation between {displaystyle x} and {displaystyle y} arises if they are not related?

Answer 55

p-value ## Footnote A significance level of {displaystyle alpha = 0.05} is used to determine statistical significance.

Question 56

True or false: A **p-value** smaller than 0.05 indicates statistical significance.

Answer 56

TRUE ## Footnote If the p-value is greater than 0.05, the coefficient is not statistically significant.

Question 57

The **t** and **F** tests are important in what type of analysis?

Answer 57

Regression analysis ## Footnote They help determine whether to accept the relationship in question.

Question 58

What do **t statistics** indicate in regression analysis?

Answer 58

Good predictor of independent variable ## Footnote They assess the significance of each independent variable.

Question 59

What do **F statistics** indicate in regression analysis?

Answer 59

Good model for regression as a whole ## Footnote They evaluate the overall significance of the regression model.

Question 60

In a regression with one independent variable, the **t** and **F** tests yield what?

Answer 60

Same result ## Footnote This indicates the relationship is assessed consistently.

Question 61

What is the **R²** value used for in regression analysis?

Answer 61

Indicator for goodness of fit ## Footnote It shows how much variability in the data is accounted for by the regression.

Question 62

Higher values of **R²** indicate what about the regression?

Answer 62

Better fit of the regression ## Footnote Values closer to 1.0 are preferred.

Question 63

What does a **p-value** of 0.11 indicate regarding statistical significance?

Answer 63

Not statistically significant ## Footnote It is greater than the significance level of 0.05.

Question 64

What should an analyst do if there is a lack of statistical significance?

Answer 64

Check the functional form ## Footnote This may involve analyzing residual plots for better models.

Question 65

The equation for **R²** is: {displaystyle R² = ?}

Answer 65

R² = explained variation / total variation ## Footnote It can also be expressed as R² = 1 - SSE/SST.

Question 66

What does **SSR** stand for in regression analysis?

Answer 66

Sum of squares due to regression ## Footnote It measures the explained variation in the dependent variable.

Question 67

What does **SST** stand for in regression analysis?

Answer 67

Total sum of squares ## Footnote It measures the total variation in the dependent variable.

Question 68

What does **SSE** stand for in regression analysis?

Answer 68

Sum of squares due to error ## Footnote It measures the unexplained variation in the dependent variable.

Question 69

The **R²** value for the toy problem is calculated as follows: R² = ?

Answer 69

R² = SSR / SST ## Footnote For the toy problem, R² = 30/48 = 0.62.

Question 70

What does a **high R²** value (greater than 0.9) indicate?

Answer 70

Strong relationship ## Footnote It suggests a good fit for the regression model.

Question 71

In regression analysis, the **total variability** is represented by which arrow in Figure 8.20?

Answer 71

Larger blue arrow ## Footnote It represents the total variability in the Y data.

Question 72

What does the **smaller orange arrow** in Figure 8.20 represent?

Answer 72

Variability after regression line ## Footnote It shows the variability in Y data after accounting for the regression.

Question 73

What does **SSR** stand for in regression analysis?

Answer 73

Sum of Squares due to Regression ## Footnote SSR is a measure of the variation explained by the regression model.

Question 74

What does **SST** represent in the context of regression?

Answer 74

Total Sum of Squares ## Footnote SST measures the total variation in the dependent variable.

Question 75

The value of the **correlation coefficient** is the square root of which statistic?

Answer 75

R² ## Footnote R² indicates the proportion of variance in the dependent variable that can be explained by the independent variable.

Question 76

True or false: The **correlation coefficient** ( r ) can be negative.

Answer 76

FALSE ## Footnote The correlation coefficient ( r ) is always positive since it is derived from a square root.