NLP Basics

Question 1

What is NLP?

Answer 1

Natural Language Processing

• Natural Language (as opposed to Artificial Language)
• Process with Computers
• Combination of Linguistics (study of languages) and Computer Science, information engineering and ai.
• Text or spoken voice
• Branch of AI that helps computers understand, interpret, and manipulate human language (SAS). Understanding and responding back.

Question 2

What do we mean by Natural Language?

Answer 2

Natural vs. Artificial Language

• Natural Language evolved gradually over time, largely unconscious and is used for daily communication between people. Complex syntax, ambiguous semantics. Contains humor, irony, metaphor, connotation, neologisms.
• Designed, crafted, invented consciously (e.g., programming language, Elvish from Lord of the Rings, Klingon from Star Trek, Esperanto for German, Italy, etc. to have an easy to learn international language, Interlingua). Usually rule-based. Readily parsed, unambiguous, subject to regular, consistent rules of interpretation.
• Interesting fact: LISP and Prolog were created for AI, but not used as much anymore
• Morse code is not an artificial language. It’s just a code for alphabet and numbers.
• Flag semaphore is not an artificial language, it’s a code.
• Braille is the same thing.
• Sign language is not an artificial language.
• Oxford English different a few hundred thousand words
• Natural Languages were not designed to be processed by machine

Question 3

What are the two sides of NLP?

Answer 3

NLP is either NLU or NLG.

• NLU: Natural Language Understanding. Trying to understand language that came from the ordinary world. This is a useful, structured interpretation of an input, like speech or written text. E.g., producing topics. We don’t care if the machine understands. It’s something useful that we can understand. Search is NLU.
• NLG: Natural Language Generation. Making natural language. We feed something like not sentences and we want the machine to output sentences. Chatbots are examples of NLG.

Question 4

Who was the first person to think about can AI exist?

Answer 4

2000 years ago. Aristotle with toys and statues. Create a create that it would fool people into thinking it was real. Aristotle said no. Liebnez in late 1800s as well.

Question 5

What are the applications of NLU?

Answer 5

Automated Text Annotation

• Tagging (important words and phrases)
• Metadata extraction/generation (data about the documen, e.g., author of a document, date)
• Classification (e.g., news article is sports, celebrities)
• Document summarization

Corpus analytics (corpus is document collection)

• Theme extraction
• Clustering
• Taxonomy mapping (mapping documents from one taxonomy to another taxonomy)
• Sentiment analysis (usually across a corpus, e.g, look at billions of tweets and find the ones that have negative emotion or cluster them)

Search applications

• Query repair (e.g., for a typo, did you mean x?)
• Query refinement (e.g., semantically ambiguous interpretation)
• Result postprocessing (ranking, clustering, encapsulation)

Advanced applications

• Machine translation
• Knowledge discovery
• Question handling (e.g., mapping questions to FAQs)

Question 6

What are the applications of NLG?

Answer 6

Often a way of make NLU more digestable to humans.

Text annotation (fancier version of NLU)
-Document summarization, e.g., make new sentence out of old sentence and make that the title of the article

Corpus analytics

• Adding labels on top of clusters (e.g., cheesy recipes, spicy recipes)
• Synopsizing corpus-wide topic and/or sentiment trends

Search applications

• Advanced capsule generation
• Advance query refinement

Advanced applications

• Machine translation
• Knowledge discovery
• Question handling (e.g., where’s a good view of Golden Gate Bridge?, getting back a question do you want a vista, hotel, restaurant, etc.)

Question 7

What is a token?

Answer 7

A token is the technical name for a sequence of characters — such as hairy, his, or :) — that we want to treat as a group.

Question 8

What is a collocation?

Answer 8

A collocation is a sequence of words that occur together unusually often. Thus red wine is a collocation,

Question 9

What is a collocation?

Answer 9

A collocation is a sequence of words that occur together unusually often. Red wine is a collocation, machine learning, social media. When you split them up, they mean something different, flat screen. NLTK has a collocations() function.

Question 10

What is a bigram?

Answer 10

A list of word pairs. NLTK has a bigrams function.

Question 11

What is a Turing test?

Answer 11

Can a dialogue system, responding to a user’s text input, perform so naturally that we cannot distinguish it from a human-generated response?

Question 12

What is RoBERTa?

Answer 12

An optimized method for pretraining self-supervised NLP systems

Question 13

What is NER?

Answer 13

Named Entity Recognition. Locating entities in unstructured text.

Question 14

What is Document Classification?

Answer 14

NLU application. Example is a spam filter. Amazon does large classification massive classification of product categories.

Question 15

What is Search: Query Understanding?

Answer 15

Inferring the intent and meaning of a search engine users queries.

Query Segmentation:  partition the queries into semantic units
Query Scoping (NER):  map query segments to entity types
Query Expansion:  broaden the query by adding additional phrases/tokens (usually synonyms and abbreviations, e.g., ML or Machine Learning, developer or engineer)
Query Relaxation:  make the query less restrictive by removing tokens (Black propane grill vs. propane grill)

Question 16

What are Dialog Systems?

Answer 16

Conversational agents, conversational ai, chatbots.

Question 17

What is machine translation?

Answer 17

NLG applications. Translate source text in one language to another. Google is the best at translation at scale. ML techniques and NLP.

Question 18

What is Document Summarization?

Answer 18

Generate accurate summaries of longer text, e.g., model-written headline

Question 19

What is lexical diversity?

Answer 19

Calculating the range of different words used in a text. A greater range = more lexical diversity.

Question 20

What is a WSD?

Answer 20

Word sense disambiguation - identifying words with different meanings (see levels of nlp for more)

Question 21

What is a polysemic word?

Answer 21

A word that has multiple meanings depending on context. “The bank will be closed this Saturday” and “The river overflowed the bank.” A lexicon tries to list word senses in order from most common to least common in occurrences.