22 Oct - Processing II (Luca)

Question 1

What do we define as outliers?

Answer 1

Data points that differ significantly from others

Question 2

What kind of impact can outliers have in machine learning?

Answer 2

KNN: Overfit partition of space
Linear regression: Very sensitive to outliers,
decreases quality of the fit
Decision trees: more robust to outliers, but can introduce overfit

Question 3

How can we deal with outliers?

Answer 3

Trimming / truncation:
drop values above/below a certain value or percentile

Winsorizing / winsorization:
setting values above/below a certain value or percentile to the closest value

Question 4

What is an isolation tree?

Answer 4

Simple yet effective anomaly detection
* Use principle similar to Decision Trees
* Algorithm:
* Pick a random split between min and max
* Count how many points are in each partition
* Points in singleton partitions are marked with the
number of splits done so far
* Repeat the split on each partition
* Stop when all points are in singleton partitions
* A point’s number of splits is its anomaly score
* The score needs normalization (see original paper)

Question 5

How to do forest isolation on n dimensions?

Answer 5

1. N-dimensional data (X1, …, Xn)
2. Pick a random dimension Xi
3. Pick a threshold t on dimension Xi
4. Split the datapoints on t
5. Repeat from point 1. recursively on
   each partition

Question 6

Why should we remember to handle outliers carefully?

Answer 6

Outliers can be:
* Noise, we want to remove it
* Originated by processes we are not interested in modeling (twitter bots)
* Originated by processes that differ from the main one we are focusing on,
but are still relevant to the analysis (power users)

Question 6

How to handle missing data?

Answer 6

• Where is it?
• Everywhere in real-world datasets
• Why missing?
• Value is unknown (e.g., missing measurement)
• Value is zero or does not apply
• What is their representation?
• Empty, None, “N/A”, “Null” …
• Who should deal with them?
• If you are performing the analysis, you should
• If you are preparing the dataset, your users should –
  but you should document what missing values mean

Question 7

WHat is the three types of missing data?

Answer 7

Missing Completely At Random (MCAR)
* Missing At Random (MAR)
* Missing Non At Random (MNAR)

Question 8

How can we deal with missing data?

Answer 8

Deletion
* Delete records with missing values
* Suitable when
* Data is Missing Completely At Random – else, bias
is introduced
* A small % of records is missing
* There is no reliable way to infer the value (see next)
* Deletion of entire column when
* A large % of records is missing
* Column is not crucial for the analysis

Single imputation – average
* Filling missing values with some global criterion
* Default value
* Fill with average/median/mode
* Suitable when
* Data is Missing Completely At Random
* A small % of records is missing
* Introduces bias in distribution
* Variability is reduced

Single imputation – local average
* Filling missing values using local information
* Applicable to data in which records are not independent,
and for which the assumption “what is close is similar”
holds
* Time series
* Audio
* Spatial data
* Images