Chapter 5 - Predictive Analytics II: Text, Web, and Social Media

Question 1

What is Text Mining?

Answer 1

The semiautomated process of extracting patterns from large amounts of unstructured data sources.

Question 2

What are the Seven (7) Application Areas of Text Mining?

Answer 2

1. Information Extraction
2. Topic Tracking
3. Summarization
4. Categorization
5. Clustering
6. Concept Linking
7. Question Answering

Question 3

What are the Fourteen (1-5) Text Mining Terms we need to know?

Answer 3

1. Unstructured Data - Data that does not have a predetermined format and is stored as textual documents.
2. Corpus - A large and structured set of texts prepared for the purpose of conducting knowledge discovery.
3. Terms - Single word or phrase extracted directly from the corpus
4. Concepts - Features generated from a collection of documents
5. Stemming - Reducing inflected words to their base or root form

Question 4

What are the Fourteen (6-10) Text Mining Terms we need to know?

Answer 4

6. Stop Words - Words that are filtered out prior to or after processing of natural language data.
7. Synonyms and polysemes - Polysemes are also called homonyms (spelled exactly the same)
8. Tokenizing - Assignment of meaning to blocks of text (also known as tokens)
9. Term Dictionary - Collection of terms specific to a narrow field that can be used to restrict the extracted terms within a corpus
10. Word Frequency - Number of times a word is found

Question 5

What are the Fourteen (11-14) Text Mining Terms we need to know?

Answer 5

11. Part-of-Speech Tagging - Marking up the words in a text as corresponding to a particular part of speech based on a word’s definition and the context in which it is used.
12. Morphology - Studies the internal structure of words
13. Term-By-Document Matrix (Occurrence Matrix)
14. Singular Value Decomposition (Latent Semantic Indexing)

Question 6

What does NLP Stand For and How is it Defined?

Answer 6

Natural Language Processing studies the problem of “understanding” the natural human language, with the view of converting depictions of human language into more formal representations that are easier for computer programs to manipulate.

Question 7

What are some of the Challenges Related to NLP? (6)

Answer 7

Part-Of-Speech Tagging
Text Segmentation
Word Sense Disambiguation
Syntactic Ambiguity
Imperfect or Irregular Imput
Speech Acts

Question 8

What is Deception Detection as it Relates to Text Mining?

Answer 8

It is used in prediction models to differentiate deceptive statements from truthful ones

Question 9

What is Part-Of-Speech Tagging?

Answer 9

Tokenized terms (words) are matched and interpreted against the text based on the term’s definition and the context that it is being used.

Question 10

What are the Three (3) Steps/Tasks for Text Mining?

Answer 10

1. Establish the Corpus - Collect all documents related to the context being studied and transform them in a manner that they are all in the same representational form for computer processing.
2. Create the Term-Document Matrix - Rows represent documents and columns represent terms. Relationships between the terms and documents are characterized by indices.
3. Extract the Knowledge - Main extraction methods are Classification, Clustering, Association, and Trend Analysis.

Question 11

What is a TDM?

Answer 11

A Term-Document Matrix that indexes the relationships between terms and documents.

Question 12

What is SVD?

Answer 12

Singular Value Decomposition reduces the overall dimensionality of the input matrix to a lower-dimensional space where each consecutive dimension represents the largest degree of variability between words and documents.

Question 13

What is Sentiment Analysis?

Answer 13

Sentiment analysis is trying to answer the question “What do people feel about a certain topic?” by digging into opinions using a variety of automated tools.

Question 14

What are the Seven (7) Discrete Sentiment Analysis Applications Stated by the Author?

Answer 14

1. Voice of the Customer (VOC)
2. Voice of the Market (VOM)
3. Voice of the Employee (VOE)
4. Brand Management
5. Financial Markets
6. Politics
7. Government Intelligence

Question 15

What is the Sentiment Analysis Process?

Answer 15

1. Sentiment Detection
2. N-P Polarity Classification
3. Target Identification
4. Collection and Aggregation

Question 16

What are the Three (3) Different Elements of Sentiment Analysis?

Answer 16

Polarity Identification
Identifying Semantic Orientation of Sentences and Phrases
Identifying Semantic Orientation of Documents

Question 17

What is Polarity Identification?

Answer 17

The process of identifying the sentiments under one of two opposing polarities, or locate the position along the continuum between the polarities.

Question 18

What are the Two (2) Methods of Polarity Identification?

Answer 18

1. Using a lexicon as a reference library
2. Using a collection of training documents as the source of knowledge about the polarity of terms within a specific domain.

Question 19

What is Web Mining?

Answer 19

The process of discovering intrinsic relationships from Web data, which are expressed in the form of textual, linkage, or usage information.

Question 20

What are Web Crawlers?

Answer 20

AKA Spiders are used to read through the content of a website automatically.

Question 21

What is an Authoritative Page?

Answer 21

Use of a web page or a relevance index that improves the search results and rankings of relevant pages.

Question 22

What is a HITS?

Answer 22

A hyperlink-induced topic search. It is a link-analysis algorithm that rates Web pages using the hyperlink information contained within them.

Question 23

What is Web Structure Mining and Why is it Important?

Answer 23

Web Structure mining is the process of extracting useful information from the links embedded in Web documents. It is used to identify authoritative pages and hubs which are the cornerstones of page-rank algorithms relied upon by Google and other search engines.

Question 24

What is SEO?

Answer 24

Search Engine Optimization is the intentional activity of affecting the visibility of a website in a search engine’s natural search results.

Question 25

What is Clickstream Analysis?

Answer 25

The analysis of the information collected by Web servers that help us better understand user behavior. It is used to discern interesting patterns from clickstreams.

Question 26

What is Social Analytics?

Answer 26

Monitoring, analyzing, measuring, and interpreting digital interactions and relationships of people, topics, ideas, and content.

Question 27

What are the Three (3) Social Network Categories?

Answer 27

Connections Distributions Segmentation

Question 28

What are the Subcategories for Connections? (5)

Answer 28

Homophily - Actors form ties with similar vs. dissimilar others Multiplexity - Number of content forms contained in a tie Mutuality/Reciprocity - How much two actors reciprocate interaction or friendship Network Closure - Measure of the completeness of relational triads Propinquity - Tendency to have more ties with geographically close others

Question 29

What are the Subcategories for Distributions? (6)

Answer 29

Bridge - An individual whose weak ties fill a structural hole, providing the only link between two individuals or clusters Centrality - Metrics that aim to quantify the importance of a particular node within a network Density - Proportion of direct ties in a network relative to the total Distance - Minimum number of ties needed to connect two actors Structural Holes - Absence of ties between two parts of a network Tie Strength - Linear combination of time, emotional intensity, intimacy, and reciprocity

Question 30

What are the Subcategories for Segmentation? (3)

Answer 30

Cliques and Social Circles - Cliques if every individual is tied to every other individual and Social Circles if there is less stringency of direct contact Clustering Coefficient - Likelihood two members of a node are associates. Higher clustering indicates a great cliquishness Cohesion - Degree to which actors are connected directly to each other by cohesive bonds