Why do we need data reduction techniques?
- Because of mammoth data sets – these datasets have loads of observations
- Only a few important features contribute to the dataset
What does data reduction allow?
- Data reduction allows us to extract the necessary information from a huge array of data and remove redundant data
- Removes noise
2 types of data reduction techniques
- Dimensionality reduction – reduce the number of input variables in a dataset
- Numerosity reduction – reduce data volume by using suitable forms of data representation
examples of dimensionality reduction
- Wavelet transform
- Attribute subset selection
- Principal component analysis
examples of numerosity reduction
histogram
sampling
clustering
wavelet transofrm
can be applied to ECG signals
• Helps to convert ECG signal into a form which makes it much easier for the QRS peak finder algorithms
Attribute subset selection
- Find a minimum set of attributes which find the same solution
- With this you have reduced cost because there is less variables
- Makes it easier for pattern recognition
Principal component analysis
- A variable reduction technique
- Reduces larger set of variables into smaller set of ‘artificial variables’ called principal components that account for most of the variance in the original variables
PCA can be used to solve 3 major problems:
- Removing unrelated variables
- Reducing redundancy in a set of variables
- Removing multicollinearity
Assumptions for PCA
- You have multiple continuous variables
- Linear relationship between all variables
- No outliers
- Large sample size for PCA to produce reliable result
Steps of principal component analysis:
- Calculate covariance matrix
- Compute the eigenvectors and eigenvalues of the covariance matrix to identify principal components
- Choose which components to keep (ones with high eigen values)
- Reorient the data from the original axes to the ones represented by the principal components using the covariance matrix
Covariance matrix
• It is a square matrix that shows covariances of each pair of variables
Principal component
- In order to perform a PCA you need to find the axis of the greatest variance which is the line of best fit
- This line is the first principal component
- Then you project your data points onto the first principal component
- Before, each person was represented by lung function and oxygen but now they are represented by one principal component
- Second principal component accounts for next highest variance
Eigenvalues and eigenvectors
- A matrix represents a linear transformation which means that a matrix contains a set of rules for moving data points around
- One type of linear transformation is shearing – where data points are sheared by multiplying them by the matrix
- Eigenvector corresponds to direction
- Eigenvalue is how far along the line the data point has moved and corresponds to distance
- Eigenvalues represent total amount of variance that be explained by a given principal component
- Ranking eigenvectors in order of their eigenvalues highest to lowest you get principial component in order of significance
Rotations
- Goal of rotation is to improve the interpretability of the factor solution by reaching a simple solution
- Rotation helps make it easier to interpret PCA
2 types of rotations:
- Orthogonal rotation (e.g., VARIMAX) – assumes components are independent or uncorrelated with each other
- Oblique rotation (e.g., direct oblimin) – components are not independent and are correlated
Putting it all together - PCA
PCA:
Calculate covariance matrix of data
calculate eigenvectors of the covariants matrix = principal components
eigenvector with largest eigen value = first principal component
eigenvalues represent the total variance that can be explained by a given principal component
