Ex Sheets

Question 1

Prove

Question 2

Prove

Answer 2

Crucially, we add a 0 vector to the bottom of the new Wⁱ and a single zero to the new bⁱ BUT Wⁱ⁺¹ adds a zero vector to the right side because the dim that increases is transposed

Question 3

Regularisation has what effect on bias variance trade off

Answer 3

Slightly increases bias (improving generalisation)

But significantly decreases variance

Question 4

L2 regularise this and then take grad of

Answer 4