Distributional Semantics

Question 1

How do we learn new words?

Answer 1

• look in a dictionary
• from experience of usage
• similar words from the past

Question 2

What is the distributional hypothesis?

Answer 2

Similar context suggests similar meanings

Question 3

In distributional semantics, we want to find f, where f is

Answer 3

a function that takes in and transforms and compresses contexts to produce a vector that encompasses the meaning of a word
meaning(w) = f(c1, c2, c3, c4)

Question 4

How do we find function, f?

Answer 4

use co-occurrence vectors

Question 5

what is a cooccurrence vector?

Answer 5

collect a corpus of documents or sentences

apply basic preprocessing like lower case

count how many times word u appears with word v

the meaning of u is vector [(count(u,v1), count(u,v2)…]

Question 6

what are the benefits of cooccurrence vectors (3)

Answer 6

• meaning of a word is vector so we can compute similarities like the cosine similarities
• can visualise word meanings
• can directly use these vectors as input to machine learning models

Question 7

what are the disadvantages of cooccurrence vectors

Answer 7

distributional semantics beyond words

cant capture all aspects of semantics