Big Data Architectures

Question 1

What is big data?

Answer 1

A collection of large and complex data sets which are difficult to process using
common database management tools or traditional data processing
applications

Question 2

What are the 4 Vs of Big data?

Answer 2

Volume -> Data at rest
Velocity -> Data in Motion
Variety -> Data in many forms
Veracity -> Data in doubt

Question 3

What are the two types of sclaling? (ability of the system to adapt to increased demands)

Answer 3

1. Horizontal scaling:
   - -> distribute workload across many servers by adding multiple machines to improve processing capacity
2. Vertical scaling
   - -> involves installing more processors, more memory and faster hardware typically within a single server (make it bigger instead of more)

Question 4

What are the advantages and disadvantages of horizontal scaling?

Answer 4

Advantages:
- increases performance in small steps as needed
- financial investment is relatively small
can scale up as much as needed

Disadvantages:

• Software has to handle all the data distribution
• There are only a limited number of softwares available than can take advantage of horizontal scaling

Question 5

What are the advantages and disadvantages of vertical scaling?

Answer 5

Advantages:
- Most softwares can easily take advantage of vertical scaling
- easy to install hardware within a single machine
Disadvantages:
- requires substantial financial investment
- system has to be more powerful to handle future workloads
- does not necessarily scale up vertically after a certain limit

Question 6

What are horizontal scaling platforms?

Answer 6

Peer to peer networks

apache hadoop

Question 7

What are vertical scaling platforms?

Answer 7

Multicore processors
HPC high performance computing clusters
Graphics processing units

Question 8

What is a peer to peer network?

Answer 8

• typically involves millions of machines connected in a network
• decentralized and distributes network architecture
• message passing interface
  -each node is capacle of storing and processing data
  scale is practically unlimited

drawbacks:
- communication is a major bottleneck

Question 9

What is apache hadoop?

Answer 9

an open source software for storing and processing large datasets

Question 10

what are high performance computing clusters? (HPC)

Answer 10

also known as super computers with throusands of processing cores

built powerful hardware optimized for speed and
throughput

Question 11

What are multicore CPUs?

Answer 11

One machine having dozens of processing

cores