Notebook 7 COPY

Question 1

What are the two main pandas data structures?

Answer 1

Question 2

import statement for pandas?

Answer 2

import pandas as pd

Question 3

How to read a csv in pandas?

Answer 3

Question 4

How do we look at the first 10 and last 10 entries in a dataframe?

What is an issue that arises here?

Answer 4

Cant view both at the same time use the name of the dataframe e.g. just “iris” to display both the first and last 5 entries in a data frame

doing this also displays the number of rows and columns in the dataframe

Question 5

How do you “print” both the head and tail of a table in the same comand line

Answer 5

Question 6

What does df.info report?

Answer 6

Question 7

What does df.describe() report?

Answer 7

Question 8

How would we pull the column “sepal_length” from the iris dataframe?

Answer 8

Question 9

What are the three ways of indexing and selecting data in a dataframe?

Answer 9

Question 10

for the iris dataframe, display two new data frames where df1 contains the columns “sepal_length” and “sepal_width” using column name indexing and the second contains displays the same using slicing?

Answer 10

Question 11

How do we print the shape of dataframe?

Answer 11

df.shape

Question 12

What are the two ways to extract the design matrix X and vector of targets y from the iris Dataframe?

Answer 12

Question 13

How can we find the mean of a df using np?

Answer 13

np.mean(df)

Question 14

How can we sort a dataframe based off as specific column in ascending order?

Answer 14

in desending order can set ascending = False or drop it from the code all together

Question 15

What do we need to check if our csv isnt not in columns?

Answer 15

Need to check how it is separated in this case it was separated by a semi-colon

Question 16

What is the import statement for seaborn?

Answer 16

import seaborn as sns

as the guys name was Samuel Norman Seaborn

Question 17

What does the following code display?

sns.pairplot(iris, hue=’species’)
plt.show()

Answer 17

Question 18

How do I adjust this function to only display the lower corner (as it is a mirror of the upper corner) and adjust the markets of the graphs)?

Answer 18

Question 19

How do you use seaborn to plot this following graph for the iris dataset?

What does it show?

Answer 19

removing the inner quartile replaces it with a boxplot

Also as seaborn likes the graphs to be based on categorical data if you swap the x and y axis the graph will be orientated horizontally.

Question 20

How do I create a heatmap for the data set “flights” with seaborn?

Answer 20

Question 21

What do we use scikit-learn for?

Answer 21

Question 22

What are the import states for some of the main ML algos we will use?

Answer 22

from sklearn import linear_model,tree

Question 23

Given our design matrix X and label vector y how can I use SVM to train and make a prediction on the sollowing unseen array?

X_unseen = np.array([[6.7, 2.8, 5.2, 2.1]])

Answer 23

Question 24

How do I adjust this code to use the decision tree classifier?

Answer 24

Question 25

What are the advantages of the decision tree classifier?

Answer 25

Question 26

What are the disadvantages of the decision tree classifier?

Answer 26

Question 27

What is the code for loading and printing a confusion matrix? What does the matrix show?

Answer 27

ADD TO THIS

Question 28

How does a decision tree classifier work?

Answer 28

ADD TO THIS from user guide

Question 29

What is the import code for train_test_split?

Answer 29

Question 30

Why dont we test on the same parameters that our model learns from?

Answer 30

Question 31

How do we create test and training data code for the iris data set using the train_test_split fucntion?

Answer 31

Question 32

When testing for different hyperparameters why is their still a risk of overfitting?

Answer 32

Question 33

How do we solve this issue?

Answer 33

Question 34

How do we implement the cross-validation procedure here?

Answer 34

Question 35

1) load diabetes data set 2) Split the target training/test data 3) Create the linear regression 4) train the model 5) Make predictions using the testing set 6) plot the target as a funciton of bmi of body mass 7) plot target as function of bp average blood preure 8) plt truth vs prediction

Answer 35

Question 36

What is the least squares solution? What is the big-O of OLS?

Answer 36

Question 37

What is the Ridge regression from OLS?

Answer 37

Has the same time complexity as OLS

Question 38

How do we implement the Ridge model?

Answer 38

Question 39

What is the classifier varient of Ridge on scikit learn?

Answer 39

Question 40

How do we set the regularisation parameter for RidgeCV and RidgeClassifier CV?

Answer 40

Question 41

What is the Lasso model?

Answer 41

Question 42

How do we implement the Lasso model?

Answer 42

Question 43

How do we set the regularisation parameters for Lasso?

Answer 43

Question 44

What else does the estimator LassoLarsIC use to find its optimal alphas?

Answer 44

Question 45

What is the equivalence between the alpha of lasso and the regularisation parameter of SVM?

Answer 45

Question 46

How do I adjust this code for the Ridge regression?

Answer 46

Question 47

How do I adjust this code for the Lasso regression?

Answer 47

Question 48

Why do we use the standardscaler?

Answer 48

Question 49

What does a confusion matrix show?

Answer 49

Question 50

What is the code to use svm to classify some data and then create a confusion matrix?

Answer 50

Question 51

What are the three way to scale the data?

Answer 51

Question 52

What does the StandardScaler?

Answer 52

Question 53

Once we have initialised scaled data, how do we implement it into code using a SVM classifier and print its confusion matrix and score?

Answer 53

Wha

Question 54

What classifier isnt that sensitive to scaling?

Answer 54

Question 55

How to implement the DecisionTreeClassifier model using scaling for the winequality date set? Perform the classification and repeat and display the confusion matrix and clf.score? How does it compare to when you use unscaled data (repeat the task with unscaled data)?

Answer 55

Not that much different

Question 56

What are the two ways to show a confusion matrix based on this unscaled train_test data of the winequality dataset?

Answer 56

Question 57

What are hyperparameters and how do they relate to Cross-validation?

Answer 57

Question 58

What is the simpliest way to use cross-validation?

Answer 58

Question 59

What happens when the cv argument is a integer in cross_val_score?

Answer 59

Question 60

How do the cross_validate and cross_val_score differ?

Answer 60

Question 61

What does cross_val_predict do? When is it appropriate or inappropriate to use?

Answer 61

Question 62

What function do we use to find the names and current values for all parameters for a given estimator?

Answer 62

Question 63

What does a search for a hyperparameters consist of generally?

Answer 63

Question 64

What are the two generic approaches for parameter searches?

Answer 64

Question 65

What is the Exhaustive Grid Search method in SciKit learn?

Answer 65

Question 66

How do we tune the decision threshold of the classifier once he model has been trained?

Answer 66

Question 67

How can we tune the decision theshold through different strategies controlled by the parameter scoring?

Answer 67

Question 68

What do you need to note regarding the internal cross-validation?

Answer 68

W

Question 69

What are all the import statements needed for cross validation?

Answer 69

Question 70

Answer 70

Question 71

Run the svm.SVC classifier on the unscaled data

Answer 71

Question 72

How do we code this?

Answer 72

Question 73

When tuning Hyperparameter seearch with a manual search with a loop what do we do once we have calculated the scores to decide which Hyperparameter is the best?

Answer 73

Question 74

How do we perform a Hyperparameter grid search using GridSearch CV? How do you perform a Hyperparameter grid search over both C and gamma at the same time?

Answer 74

If things have gone well, you should find that with C and gamma set you can now predict approximately 66% of the cases correctly. This is a substantial improvement over the approximately 45% found at the beginning of this notebook before doing any scaling or hyperparameter tuning. Note also that the 66%

Question 75

What do we do once we have the best set of hyperparameters?

Answer 75

Question 76

What is the default number of folds the GridSearchCV() uses

Answer 76

cv = 5