Dataset Preparetion

Question 1

Missing Data Imputation Strategy: Mode

Answer 1

Replace empty values with the most frequent value in the same column.

Question 2

Train-Test Split

Answer 2

Dividing a dataset into two separate sets: one for training a model and another for evaluating its performance.

Question 3

Data Imputation

Answer 3

Replacing missing entries with estimated values based on statistical measures or model predictions.

Question 4

Algorithms that DON’T Need Normalization

Answer 4

Decision Trees and Random Forest.

Question 5

Inspect Dataset Structure and Statistics: Steps

Answer 5

Load the Data. Review Shape of Data. Display Data Types. Look at Basic Stats to Grasp Information Available.

Question 6

Procedure to get the data types and non-null counts of a Pandas Dataframe

Answer 6

Use the command $print(df.info())$

Question 7

Goal of Feature Selection

Answer 7

Keeping only the most informative features and discarding irrelevant or redundant ones to improve model performance.

Question 8

Normalization Range

Answer 8

Scales data to a fixed interval, typically [0,1].

Question 9

Formula for calculating normalized value using min-max scaling

Answer 9

$x_{norm} = \frac{x - x_{min}}{x_{max} - x_{min}}$

Question 10

Cross-Validation

Answer 10

A technique for assessing how well a model generalizes to an independent dataset by training and validating on different subsets of the data.

Question 11

Algorithms that Need Normalization

Answer 11

Algorithms using distance metrics (KNN, K-Means) and gradient descent.

Question 12

Missing Data Imputation Strategy: Mean

Answer 12

Replace empty values with the average of the other values in the same column.

Question 13

When to Remove Duplicate Rows

Answer 13

Unless duplicates represent meaningful replications, get rid of them.

Question 14

Python library commonly used for data manipulation and analysis

Answer 14

Pandas

Question 15

Label Encoding

Answer 15

Converting categorical labels into numerical values, assigning a unique integer to each category.

Question 16

Encode Categorical Features and Scale Numeric Ones: Steps

Answer 16

Encode Categorical Columns. Apply Normalization or Standardization to Numeric Columns.

Question 17

Duplicate Detection

Answer 17

The process of identifying and locating identical rows within a dataset that could potentially skew the analysis.

Question 18

Duplicate CustomerID Rows

Answer 18

Remove them.

Question 19

Procedure to display the number of rows and columns of a Pandas Dataframe

Answer 19

Use the command $print(df.shape)$

Question 20

One-Hot Encoding

Answer 20

Converting categorical data into a binary matrix, creating a new column for each category.

Question 21

Variance Thresholding Purpose

Answer 21

Removes low-variance variables.

Question 22

Python library commonly used for machine learning algorithms

Answer 22

Scikit-learn

Question 23

Missing Data Imputation Strategy: Median

Answer 23

Replace empty values with the middle value of the other values in the same column.

Question 24

Procedure to view the first few rows of a Pandas Dataframe

Answer 24

Use the command $print(df.head())$

Question 25

Standardization (Z-Score Scaling) Formula

Answer 25

$z = \frac{{x - \mu}}{{\sigma}}$

Question 26

Standardization

Answer 26

Transforming data so it has a mean of 0 and a standard deviation of 1.

Question 27

Scikit-learn

Answer 27

A Python library providing machine learning algorithms and tools for model training, evaluation, and more.

Question 28

Formula for calculating standardized value using Z-score scaling

Answer 28

$z = \frac{x - \mu}{\sigma}$

Question 29

Normalization (Min-Max Scaling) Formula

Answer 29

$x_{norm} = \frac{{x - x_{min}}}{{x_{max} - x_{min}}}$

Question 30

Procedure to perform cross-validation

Answer 30

Use the command $scores = cross_val_score(rf, X, y, cv=5)$

Question 31

Procedure to get summary statistics for numeric columns of a Pandas Dataframe

Answer 31

Use the command $print(df.describe())$

Question 32

Procedure to drop duplicate rows in Pandas Dataframe

Answer 32

Use the command $df_clean = df.drop_duplicates()$

Question 33

Feature Importance

Answer 33

A measure of how significant each variable is in predicting the target variable; often visualized with charts.

Question 34

Procedure to count the number of missing values per column of a Pandas Dataframe

Answer 34

Use the command $print(df.isnull().sum())$

Question 35

Procedure to perform label encoding of a categorical column 'Size' in Pandas Dataframe

Answer 35

Use the command $df['Size_encoded'] = le.fit_transform(df['Size'])$

Question 36

Overfitting

Answer 36

A situation where a model learns the training data too well, resulting in poor performance on new, unseen data.

Question 37

Procedure to Load a CSV file with Pandas

Answer 37

Use the command $df = pd.read_csv('your_dataset.csv')$

Question 38

Procedure to perform one-hot encoding of a categorical column 'Gender' in Pandas Dataframe

Answer 38

Use the command $df_onehot = pd.get_dummies(df, columns=['Gender'])$

Question 39

Feature Selection

Answer 39

The process of choosing a subset of relevant variables for use in model construction.

Question 40

Procedure to fill the missing values of 'Age' column with median

Answer 40

Use the command $df['Age'].fillna(df['Age'].median(), inplace=True)$

Question 41

Pandas

Answer 41

A Python library providing data structures and data analysis tools.

Question 42

Missing 'Age' Value Best Approach

Answer 42

Impute it with the median or mean.

Question 43

Target Variable

Answer 43

The column in a supervised learning dataset that the model aims to predict.

Question 44

Why Split Data Into Training and Test Sets

Answer 44

Evaluate the model on unseen data to prevent overfitting.

Question 45

Dataset Inspection

Answer 45

The initial process of examining data to understand its structure, types, and basic statistical properties.

Question 46

Dataframe

Answer 46

A two-dimensional labeled data structure with columns of potentially different types.

Question 47

Handle Missing Values and Duplicate Rows: Steps

Answer 47

Identify Missing Values. Impute Missing Values. Locate Duplicates. Remove Duplicates.

Question 48

Standardization Values

Answer 48

Transforms data to have a mean of 0 and a standard deviation of 1.

Question 49

Procedure to split the data into training and test sets

Answer 49

Use the command $X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)$

Question 50

Procedure to view duplicate rows in Pandas Dataframe

Answer 50

Use the command $print(df[df.duplicated()])$

Question 51

Normalization

Answer 51

Rescaling data values to fit within a specific range, often between 0 and 1.

Question 52

Drawback of Many Irrelevant Variables

Answer 52

Increases computational cost and may hurt accuracy.

Question 53

Visualize Variable Importance with Random Forests: Steps

Answer 53

Train a Random Forest. Extract Variable Importances. Plot Importances to Rank Variables.