4.11 Big Data

Question 1

What is meant by Big Data

Answer 1

A catch-all term for data that does not fit into usual containers and so requires other techniques to process

Question 2

What are the 3 attributes of Big Data and what do they entail

Answer 2

• Volume: This means big data is too big to fit into a single server
• Velocity: Data changes very regularly and needs to be processed within a short amount of time
• Variety: Data is made up from different forms

Question 3

What are the problems with Big Data regarding volume

Answer 3

• Processing has to be carried out across multiple machines
• Results of processing can vary if multiple machines operate on connected data

Question 4

What are the problems with Big Data regarding velocity

Answer 4

• As data is spread over many locations, the time to communicate may affect the processing that can happen

Question 5

What are the problems with Big Data regarding variety

Answer 5

• It is difficult to determine the structure of data
• Traditional storage methods are not appropriate as data does not fit into table format

Question 6

What are the techniques to overcome Big Data problems that have to do with volume

Answer 6

• Programs using functional programming can be run separately in isolation without worrying about any processing side effects

Question 7

What are the techniques to overcome Big Data problems that have to do with variety

Answer 7

Data can be represented as a fact-based model to avoid a structure that is too rigid.
Links between data can be determined using a graph schema rather than a rigid structure

Question 8

What are the features of a fact based model

Answer 8

• Data is stored at individual facts with timestamps
• Each fact is immutable
• Facts can be updated with a new fact with a new data timestamp

Question 9

What are the advantages of fact based modelling

Answer 9

• Because data is immutable it reduces the risk of losing data due to human error
• It has no need for indexing because data is just appended to the dataset