TensorFlow

Question 1

Explain the SoftMax function

Answer 1

Turns scores(integers which reflect output of neural net) into probabilities

Question 2

What are logits?

Answer 2

Scores/Numbers. For neural nets, its result of the matmul of weights,input + bias. Logits are the inputs into a softmax function

Question 3

Explain a TensorFlow Session

Answer 3

• an environment for running a graph
• in charge of allocating the operations to GPU(s) and/or CPU(s), including remote machines
• Sessions do not create the tensors, this is done outside the session. Instead, Session instances EVALUATE tensors and returns results

Question 4

Explain tf.placeholder

Answer 4

• Tensor whose value changes based on different datasets and parameters. However, this tensor can’t be modified.
• Uses the feed_dict parameter in tf.session.run() to set the value of placeholder tensor
• tensor is still created outside the Session instance

Question 5

How to set multiple TF.Placeholder values

Answer 5

Question 6

What happens if the data passed to the feed_dict doesn’t match the tensor type and can’t be cast into the tensor type

Answer 6

ValueError: invalid literal for

Question 7

How to cast a value to another type

Answer 7

tf.subtract(tf.cast(tf.constant(2.0), tf.int32), tf.constant(1))

Question 8

Explain Tf.Variable

Answer 8

• remembers its a capital V
• creates a tensor with an initial value that can be modified, much like a normal Python variable
• stores its state in the session
• assign variable to tf.global_variables_initializer(), then initialize the state of the tensor manually within a session. Or call tf.global_variables_initializer() directly in the session instance.

Question 9

TF.normal

Answer 9

• The tf.truncated_normal() function returns a tensor with random values from a normal distribution whose magnitude is no more than 2 standard deviations from the mean.
• Since the weights are already helping prevent the model from getting stuck, you don’t need to randomize the bias

Question 10

Softmax function call?

Answer 10

x = tf.nn.softmax([2.0, 1.0, 0.2])

Question 11

Answer 11

Question 12

Explain steps to one-hot encode labels

Answer 12

1. import preprocessing from sklearn
2. create an encoder
3. encoder finds classes and assigns one-hot encoded vectors
4. transform labels into one-hot encoded vectors

Question 13

basic concept of cross entropy

Answer 13

calculates distances of two vectors. Usually comparing one-hote encoded vector and softmax output vector. Basic idea is to reduce distance

Question 14

Describe process of calcaulting cross entropy

Answer 14

1. take natural log of softmax outputs vector (prediction probabilities)
2. Next, multiply by one hot encoded vector
3. Sum together, take negative
4. Since one hot-encoded vector has zeros except the true label/class, the formulat simplifies to natural log of prediction probabilities

Question 15

Does the cross entropy function output a vector of values?

Answer 15

No, just a single value which represents distance

Question 16

Quiz - Cross Entropy

Answer 16

Question 17

how to implement mini-batching in TF

Answer 17

• use range to specify starting, ending and step size
• identify end of each batch
• select data and loop

Question 18

Quiz - Set features, labels, weights and biases

Answer 18

Question 19

What is a tensor

Answer 19

any n-dimensional collection of values

scaler = 0 dimension tensor

vector = 1 dimension tensor

matrix = 2 dimension tensor

Anything larger than 2 dimension is just called a tensor with # of ranks

Question 20

Describe a 3 dimensional tensor

Answer 20

an image can be described by a 3 dimensional tensor. This would look like a list of matrices

Question 21

TF.Constant

Answer 21

value never changes

Question 22

Best practice to initialize weights

Answer 22

truncated normal takes a tuple as input

Question 23

Best practice to initialize bias

Answer 23

Question 24

What does “None” allow us to do?

Answer 24

None is a placeholder dimension. Allows us to use different batch sizes

Question 25

How to implement mini\_batching in TF

Answer 25

Question 26

How to implement Epochs in TF

Answer 26

Nest loop of batches inside loop of training cycles

Question 27

How to save your trained weights and biases in TF

Answer 27

create the class using tf.train.saver run the save method on this class with the directory

Question 28

When saving weights and biases, what some considerations when applying a name

Answer 28

Basically, if weights and biases are specified in differenet order within the pipeline, and no name is specified when saving, TF will throw an error.

Question 29

How to one hot encode labels using TF function

Answer 29

Question 30

template code to calculate model softmax, cross\_entropy, specify loss and optimize

Answer 30