recurrent neural nets

Question 1

What are the applications of recurrent neural nets?

Answer 1

• Speech recognition
• Text recognition
• Stock prices
• Sequential data

Question 2

Vanilla supervised learning and Structured Input

Answer 2

• vanilla supervised models (feedforward nets) assume no input structure. These models are not good for analyzing organized info like sentences
• Doesn’t care about the order of the info

Question 3

ways to model ordered sequences?

Answer 3

• recursive functions. we want given a sequence input model it as a recursive function
• First approach is to model the recursive function as a linear function and try to fit the weighs. Do this with minimized least squares sum. This is a linear regression

Question 4

Using keras for creating a recursive linear funtion

Answer 4

• Use the dense layer with activation linear

Question 5

logistic recursive sequence

Answer 5

s​1=α

s​2​​ =ws​1​​ −ws​1^2

Question 6

What is windowing?

Answer 6

• The process of processing the sequence for training

- Obtaining the input - output pairs

Question 7

Flaws with FNN approach

Answer 7

• The probabilistic interpretation of least squares is that input/output pairs are independent and identically distributed. This is not good because it means data don’t have structure
• With iind changing a previous value the forward values don’t change
• We make things independent the opposite we want