Sentence Segmentation

Question 1

Describe the task of sentence segmentation

Answer 1

determine boundaries between sentences

Question 2

Describe the ambiguity problem in sentence segmentation

Answer 2

There are many different eos characters and styles.
e.g. “xyz”. in UK but “xyz.” in America
If we are thinking of medical writing, a sentence may not begin with a capital letter if it is a protein or something

Question 3

Describe the approaches to sentence segmentation (5)

Answer 3

• regular expressions
• dictionaries of abbreviations
• hand crafted rules
• statistical and ml
• hybrid

Question 4

Give 2 example rules for sentence segmentation

Answer 4

• Check first character after potential eos

- dictionary of abbreviations

Question 5

What tools are available for sentence segmentation

Answer 5

openNLP sentence detection (ml based)

Spacy (ml or rule based)

Question 6

What is domain dependence

Answer 6

This is the idea that the use of language is different based on its domain, e.g. medical writing