Factor Analysis

Question 1

Basic EFA process

Answer 1

• pre-analysis checks
• extraction
• rotation
• naming

Question 2

Basic concepts

Getting the name right

Answer 2

• Exploratory Factor Analysis (EFA) is really Principal Components Analysis (PCA)
• It is a mathematical technique by which patterns of correlations can be explained by a smaller number of variables (components/factors)
• It is usually (though not always) assumed that these components (factors) are uncorrelated.
• Factor analysis is a technique by which it is possible to ‘model’ the extent to which measured variables and their co-variability can by explained by a smaller number of latent variables.
• It can be exploratory…
• Often, though not always reaches the same solution as PCA
• Increasingly it is confirmatory…
• The researcher specifies a model that is then tested
• The researcher specifies several different models that are then compared in terms of the ‘goodness of fit’

•This lecture on EFA is specific to the most commonly used technique, i.e. PCA

Question 3

Basic concepts

Latent variable / factor

Answer 3

• Not directly observed, cannot be directly measured
• E.g. cognitive ability
• Measure different aspects of it
• E.g. sentence completion, computation, comprehension, spatial rotation, working memory, etc…

•Are those aspects driven by the same underlying variable?

Question 4

Basic concepts

Exploratory Factor analysis

Answer 4

• Understand structure of set of variables
• Try to find ‘simple’ structure
• identify relatively independent clusters of variables – reduce large set of variables to smaller subset while keeping information

Question 5

Overview of EFA process

Answer 5

• pre-analysis checks
• correlation matrix (R-matrix)
• sample size, number of items, etc.
• extraction
• how many factors?
• rotation
• how to best view the solution?
• naming
• What names for the factors? – how good is our EFA?

Question 6

Pre- analysis checks

Answer 6

• Some sample size pre-conditions are needed because correlations coefficient are less reliable for small samples
• N (participants) / P (items): 5:1, 10:1
• absolute minimum 100
• P (items) / M (factors) : 4:1
• N (subjects) / M (factors): 6:1
• Two Statistical Checks
• Sampling Adequacy – the extent to which the data is suitable for EFA (is there some common variability that can explained by some factors)
• Kaiser Meyer Olkin (KMO) >.5
• Is the R-Matrix an identity matrix

Question 7

Pre analysis checks

Answer 7

• Variability has three components
• Unique - Specific to that variable
• Common - Shared with other variables
• Error – Random variability
• Communality
• Proportion of variance explained by extracted factors
• A measure of ‘common’ variance.
• Communality & Sample Size
• All communalities >.6: N≥100
• Communalities ≈.5 & only a few factors: N∈[100, 200]
• Communalities <500

Question 8

Factors =

Answer 8

Linear combinations of variables

Question 9

Factor extraction

Answer 9

• There are three commonly used way to extract factors (in increasing order of reliability)
• K1 rule
• Scree test
• Parallel Analysis
• All are based on eigenvalues
• Eigenvalues are a measure of the variance explained by a factor (principal component)
• It is assumed that the more the variance that is explained the better….

Question 10

K1 rule

Answer 10

•Select all factors with eigenvalues > 1

Question 11

Scree Test

Answer 11

• Plot the eigenvalues against the component number (see previous slide)
• Cattell (1966) argued that the cut off point should be before the point of inflexion
• In this case there are two (probably – see red lines) factors
• This method can be a bit of problem if the plot doesn’t have a clear ‘point of inflexion’

Question 12

Parallel Analysis

Answer 12

• Generate a set of ‘random’ eigenvalues given N (number of participants) and P (number of items)
• Extract as many factors as there are observed eigenvalues greater than the random eigenvalues

Question 13

Factor rotation

Answer 13

• There are two types of rotation
• Orthogonal
• Oblique
• Both are attempts to achieve ‘simple structure’ (the maximization of loadings on one factor while minimizing on the other factors).
• Orthogonal Rotation
• Assume that the factors are not correlated with each other…
• Varimax rotation is one kind of orthogonal rotation
• Oblique Rotation
• Assume that the factors are correlated with each other…
• Direct Oblimin rotation is one kind of oblique rotation
• Generally more difficult to justify since in EFA it is assumed that the correlations between factors are all the same size (just not zero).

Question 14

Reliability of a factor

Answer 14

• Reliability
• A reliable measure consistently reflects the measured construct.
• Internal Reliability
• Is the measure consistent within itself.
• Split half test.
• Cronbach’s alpha (∈[-∞, 1], α≥.7)
• KR-20 (∈[0, 1], KR20≥.9
• External Reliability
• does measure vary from one use to another?
• test-retest: r > 0.7

Question 15

Naming of factors

Answer 15

• Factor Naming
• Once the factors have been identified it’s time to decide what they ‘represent’.
• Theoretical considerations
• Size of factor loading
• Common sense
• Raters
• Recaptured Item Technique (Meehl et al 1971)
• Sometimes the names can feel like they have been ‘picked out of the air’
• “The four authors, working first independently and then in conference, interpreted and named each factor by examining half the items showing high loadings on it. These 40 factor names were then presented to 10 skilled clinical judges in two batches of 20, together with sets of the other half of high-loading items per factor that had not been scrutinized in the factor-interpreting stage. The judges’ task was to do a 20 × 20 matching of factor names with item sets. Success in “recapturing” items from factor names was almost perfect, indicating that the factor interpretation was communicating valid intersubjective knowledge” (pg. 171)

Question 16

Problems with Factor analysis

Answer 16

• No way to test the solution…
• Now that fast computing is easily available it is increasing rare to rely on EFA.
• Researchers typically now move on to ‘Confirmatory Factor Analysis’ where theoretically specified models are tested directly.
• Infinite solutions…
• There are infinite rotations accounting for the same variance.
• GIGO
• Garbage In Garbage Out