Natural Language processing

Question 1

Define natural language processing

Answer 1

goal is to make machines to understand and interpret human language the way it is written or spoken

Question 2

What are different levels of linguistic analysis used in NLP

Answer 2

• syntax – what part of given text is grammatically right
• semantics – what is the meaning of given text

Question 3

What does NLU do

Answer 3

tries to understand the meaning of given text – the nature and structure of each word inside text must be known for
NLU

Question 4

What are applications of natural language understanding

Answer 4

• Search and Information Retrieval
• Enhancing search results by identifying key information, like people or locations, for better relevance.
• Word Prediction
• Predicting the next word in a sentence based on context and prior words.
• Text Classification
• Categorising text into predefined categories (e.g., spam vs. not spam, topic classification).

Question 5

What is sentence segmentation

Answer 5

identify sentence boundaries in the given text, i.e., where one sentence ends and
where another sentence begins; sentences are often marked ended with punctuation mark ‘.’

Question 6

What is tokenisation

Answer 6

identify different words, numbers, and other punctuation symbols

Question 7

What is stemming

Answer 7

strip the ending of words like ‘eating’ is reduced to ‘eat’

Question 8

What is part of speech (POS) tagging

Answer 8

assign each word in a sentence its own part-of-speech tag such as designating
word as noun or adverb

Question 9

What is named entity recognition

Answer 9

identify entities such as persons, location and time within the documents

Question 10

What is co reference (discourse) resolution

Answer 10

define the relationship of the given word in a sentence with previous and
next sentence

Question 11

What does stemming do

Answer 11

try to find the base form of words

Stemming usually refers to a crude heuristic process that chops off the ends of words
in the hope of achieving this goal correctly most of the time, and often includes the removal
of derivational affixes.

Question 12

What does lemmaliser do

Answer 12

Lemmatization usually refers to doing things properly with the use of a vocabulary and
morphological analysis of words, normally aiming to remove inflectional endings only
and to return the base or dictionary form of a word, which is known as the lemma.

Question 13

What are stop words

Answer 13

Stop words usually refer to the most common words such as “and”
“the”, “a” in a language,
but there is no single universal list of stopwords. The list of the stop words can change
depending on your application.

Question 14

What is part of speech

Answer 14

The part of speech explains how a word is used in a sentence. There are eight main parts of speech - nouns,
pronouns, adjectives, verbs, adverbs, prepositions, conjunctions and interjections.

Question 15

WHat is bag of words

Answer 15

Any information about the order or structure of words is discarded. That’s why it’s called
a bag of words. This model is trying to understand whether a known word occurs in a
document, but don’t know where is that word in the document.
The intuition is that similar documents have similar contents. Also, from a content, we can
learn something about the meaning of the document.

Question 16

What is Term Frequency(TF)

Answer 16

a scoring of the frequency of the word in the current document.
This part measures how often a word appears in a document.
The more frequently a word appears, the higher its TF score for that document.

Question 17

what is inverse Term. frequency

Answer 17

a scoring of how rare the word is across documents.
This part measures how unique or rare a word is across multiple documents in the corpus.
The rarer the word, the higher its IDF score.

Question 18

What does N-Gram do

Answer 18

basic idea underlying the statistical approach to word prediction is to use the probabilities of SEQUENCES OF
WORDS to choose the most likely next word

Question 19

What is the Markov assumption

Answer 19

only prior local context – the last few words – affects the next word
* making the Markov assumption for word prediction means assuming
that the probability of a word only depends on the previous N-1 words
(N-GRAM model)