M18 - Factor & Cluster Analysis

Question 1

Factor and Cluste Analysis

• are … … techniques
• difference

Answer 1

• are DATA REDUCTION techniques
• difference:
  FA - groups variables
  CA groups observations

Question 2

Factor Analysis method

Answer 2

– classification of similar variables measuring the same in factors/groups

Question 3

Cluster Analysis method

• classification
• principle

Answer 3

– classification of similar objects in groups

• principle: homogeneity within each group, heterogeneity between the groups

Question 4

Factor analysis

• what is a factor?
• Factor analysis tries to identify…
• always fewer … than …
• Factor analysis is an … technique in which all the variablesare … considered, each … to all others.
• -> contrast to regression analysis

Answer 4

• factor = is a construct, hypothetical entity, ‘latent’ variable, that is assumed to underly tests
• Factor analysis tries to identify a set of common underlying factors, in a group of variables
• always fewer FACTORS than VARIABLES
• Factor analysis is an INTERDEPENDENCE technique in which all the variablesare SIMULTANEOUSLY considered, each RELATED to all others.
  – In contrast, regressions are dependence techniques in which one variable isexplicitly regarded as the dependent variable.

Question 5

What the factor laoding?

Answer 5

the correlation coeff ajs between factor and variable

Question 6

Purpose of factor analysis

1-4

Answer 6

1. test validity of a scale
2. reveal interesting patterns
3. solving problems of ulticollinearity, if two variables are correlated and theoretically meaningful
4. smaller number of variables to work with

Question 7

two types of factor analysis
1.
2.

Answer 7

1. Exploratory factor analysis
   - used to identify complex interrelationships among items and group items that are part of unified concepts. - no a priori assumptions about relationships among factors.
2. Confirmatory factor analysis
   - tests the hypothesis that the items are associated with specific factors

Question 8

Exploratory Factor Analysis

• uncovers …
• priori ass.:
• shows … and assesses …
• needs to be done …

Answer 8

• uncovers underlying structures in a large set of variables
• priori ass.: any indicator ay be associated with any factor
• shows (uni)dimensionality and assesses reliability of a scale
• ALWAYS needs to be done

Question 9

Confirmatory Factor Analysis

• determines
• … … are selected on the basis of … … and factor analysis is used to if they … as ….
• shows … …
• not … if the scale has been … …

Answer 9

• determines the number of factors based on what is expected from previous research
• INDICATOR VARIABLES are selected on the basis of PRIOR THEORY and factor analysis is used to see if they LOAD as PREDICTED
• shows GENERAL VALIDITY of a scale
• not NECESSARY if the scale has been used before

Question 10

3 steps of exploratory factor analysis
1-3

Assumptions!

Answer 10

1. Correlation matrix
   - -> examine corr of variables and KMO-criterion: should be >=0.8, must be >=0.5
2. Factor extraction from variables
3. Factor rotation
   - -> to maximize relship between variables and factors

Assumptions:

• variables continuous
• variables normally distributed
• but also for ordinal var possible

Question 11

Factor extraction method
1-2
Key differences

Answer 11

1. Principal component f.a.
   - -> maximize expl variance among underlying variables
   purpose: derive a small number of linear combinations (principal components) to retain as much information from the original variables as possible
2. Common f.a.
   - -> maximize the underlying correlations among the underlying variables

differences:
- ass Principal c.f.a.: all variance can be explained –> use: data reduction
- Common f.a.: aims at latent constructs

Question 12

How many factors?

3 criterias

Answer 12

1. Kaiser criterion: Eigenvalue >1
2. achieving a high specified cumulative % of total variance (usually 60%)
3. Elbow criteria in screeplot

Question 13

Kaiser criterion:
Eigenvalue
Communality
Factor loadings

Answer 13

Eigenvalue = sum of suqared factor loadings of one factor over all variables / the amount of variance explained by a factor
Communality = sum of squared factor laodings of one variable / the proportion of common variance present in a variable (rest is random variance)
Factor loadings = correlation of factor and variable

Question 14

Factor distinct

• why?
• what it does:

Answer 14

• so that variables only load on one factor, not many
• reference axes of the factors are turned about the origin
  – Process of adjusting axes to achieve a more meaningful factor solution

Question 15

Factor rotation

• why?
• how?
• result

Answer 15

• to discriminate between different factors
• rotate the axes such that variables are oaded maximally to only one factor
• by rotating the axes we ensure that both clusters of variables are intersected by the factor to which they relate the most
• -> after rotation the laodings of the variables are maximized on one factor

Question 16

Kaiser-Meyer-Olkin criteria

Answer 16

Exploratory Factor Analysis - Corrleation matrix
is this data set suitable for factor analysis?

• -> between 0 and 1
• -> if 0, than the sum of partial correlations is large relative to the sum of correlations –> diffusion in the pattern of correlations –> f.a. inappropriate

–> if 1, than the patterns are relatively compact and f.a. should yield distinct and reliable factors

Question 17

Bartlett’s test

Answer 17

tells us whether our correlation matrix is significantly different from an identity matrix

–> used to test if k samples are from populations with equal variances.

Equal variances across populations is called homoscedasticity or homogeneity of variances.

Question 18

What to do with the results of factor rotation?
–> study the … …

• what if a variable still shows it?
• assess reliability through … …

Answer 18

• -> study the CROSS LOADINGS
• variable should only load on one factor
• factor loading for variable of interets > 0.4 (<0.4 all others)
• drop variable and run again
• still use the factor scores
• assess reliability of a scale through CRONBACHS ALPHA
  = determines internal consistency
  – Ranges between 0 and 1. Values ≥ 0.7 are good (and expected!)

Question 19

Factor score

index scale

summated scale

Answer 19

• estimated values of the factors for each observation
• Index scales are calculated as the average score across items, summated scales as their sum
• Summated scales are a collection of related questions that measure underlying constructs.

Question 20

Difference between factor and index score?

Answer 20

index score: add the values up

factor score: add them up with a weight accord. to their factor loading

Question 21

Cluster Analysis

• goal
• cluster algorithms

Answer 21

• goal: assign observations to homogeneous clusters
• -> classification of similar objects in groups
• centroid-based / k-means: no. of clusters k is specified in advance, and then found per mathematical optimization
  – Connectivity-based or hierarchical: algorithms connect items based on their “distance” from one another.

-

Question 22

partly new algorithms exits in

Answer 22

• Big data: needs different algorithms to overcome performance issues

– Social network analysis: usually uses connectivity-based logic

Question 23

Steps in hierarchical cluster analysis

Answer 23

1. Choose a metric
   - Similarity
   - Dissimilarity –> Euclidian Distance
2. choose linkage criteria
   - -> Divisive (Top-down): start with one cluster, that is split up until no. of clusters = no. of observations
   - -> Agglomerative (Bottom-up): start no. of clusters = no . of objects, than you aggregate them until you have one cluster left
3. Choose no. of clusters
   - -> Statistically (quality measures, stopping value)
   - -> Optically (elbow criteria)
   - -> Conveniently (ease of use, heterogeneity)
   - -> Robust (check with rules of k-means clustering)

Question 24

Wrap-up Cluster and Factor analysis

Answer 24

 Factor analysis reduces the no. of variables
– allows to address issues of multicollinearity
– Key method to assess reliability and validity of constructs;

 Cluster analysis reduces the number of observations
– More practical than research applications

Question 25

Squared Euclidian DIstance

Answer 25

for each pair of objects, the difference values of each attribute are squared and added up - -> large values get higher significance - -> small differences indicate a cluster

Question 26

M19 - 1. Cluster combined 2. Coefficients 3. Stage cluster first appears and next stage

Answer 26

1. whcih clusters are merged to a new cluster in the different iterations - -> when two clusters merge always retain the lower cluster 2. the error sum of squares corresponding to the clustering iterations 3. in which iteration cases and clusters are merged with already existing clusters