Transformers

Question 1

CBOW steps

Answer 1

From input vector take embeddings of cont4xt words

Average the context words v’

Multiplying this v’ vectory with output matrix

Get the score and apply softmax

Learn from result and tune the input matrix

Question 2

Transposed convolution feature out dimension calculation

Answer 2

(i-1)*s+k-2p

Question 3

Limitations of one hot encoding

Answer 3

Produces sparse, high dimensional vectors and captures no semantic relationship

Question 4

When is TF - IDF vector outperform bag of words

Answer 4

When the distinguished power of a word is crucial and common words are noisy.

Question 5

Trade off between CBoW and Skip gram

Answer 5

CBOW fasteand better for frequent words
Skip gram is slower but better for rare words