Lecture 9 - Cloud Computing, Big Data, Tensor-flow, Recurrent Neural Networks, Distributed Deep Learning

Question 1

What is cloud computing?

Answer 1

A model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management or service provider interaction.

Question 2

What is cloud computing composed of?

Answer 2

5 characteristics, 4 deployment models and 3 service models

Question 3

What are the five characteristics of cloud computing?

Answer 3

1. On-demand self-service: Provision computing resources as needed without requiring human interaction with each service provider
2. Broad network access: Resources are available over network and accessed through standard mechanisms
3. Resource Pooling: Provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model (sense of location independence)
4. Rapid Elasticity: Capabilities can be elastically provisioned and released, to scale rapidly outward and inward as per demand
5. Measured Service: Control and optimize resource use by metering capability
   - Usages are monitored, controlled and reported -> Transparency

Question 4

What are the four deployment models of cloud computing?

Answer 4

1. Private Cloud: Exclusive use by single organization comprising multiple consumers.
   - May be owned/managed, and operated by orgaization/3rd party/combination (exists on or off premises)
2. Community cloud: Exclusive use by a specific community of consumers from organizations that have shared concerns
3. Public cloud: Use by general user.
4. Hybrid cloud: Composition of two or more distinct cloud infrastructures (Private, community or public)
   
   • Bound together by standardized technology that enables data and application portability
   • E.g., Cloud bursting for load balancing between clouds

Question 5

What are the three service models of cloud computing?

Answer 5

1. Software as a Service (SaaS): Capability provided to consumers is to use provider’s applications running on Cloud. Applications are accessible from various client devices (Example: Dropbox)
2. Platform as a Service (PaaS): Capability provided to consumers is to deploy onto Cloud consumer-created or acquired applications created using programming languages, libraries, services and tools supported by provider (Example: MS Azure Cloud, Google Cloud Platform, AWS)
3. Infrastructure as a Service (IaaS): Capability provided to consumers is to provision processing, storage, networks and other fundamental computing resources where consumer can deploy and run arbitrary software including OSs and applications

Question 6

What is the difference between traditional data and big data?

Answer 6

Big Data refers to the inability of traditional data architectures to efficiently handle new datasets.

Question 7

What are the characteristics of big data? (4 V’s)

Answer 7

• Volume (Size of dataset)
• Variety (Data from multiple sources/types)
• Velocity (Rate of flow)
• Variability(Change in other characteristics)

Bonus: (New V)
- Value

(There are different ideas about how many V’s there are. Our teacher sticks to 4(5) V’s)

Question 8

True or False: Machine Learning Techniques are very capable of processing large raw data

Answer 8

True!

Question 9

Big data require a ____ architecture for efficient storage, manipulation and analysis

Answer 9

Big data require a SCALABLE architecture for efficient storage, manipulation and analysis

Question 10

What does Data Science try to extract from data? And how?

Answer 10

Data science tries to extract: Actionable knowledge / Patterns

Through: Discovery or hypothesis formulation and hypothesis testing

Question 11

What are the six primary components of the Data lifecycle management system? (DLMS)

Answer 11

Comprises of six primary components:
- Metadata management (Maintains static and dynamic characteristics of data. I.e data about the data)

• Data placement (handles efficient data placement and data replication while satisfying user requirements - I.e will it be placed on your premises or on the cloud?)
• Data storage (is responsible for efficient(transactional) storage and data retrieval support - Key value storage/other)
• Data ingestion (enables importing and exporting data over respective system)
• Big data processing (supports efficient and clustered processing of big data by executing main logic of user application(s) - i.e how do we process the data)
• Resource management (is responsible for proper and efficient management of computational resources)
• Not necessarily part of Data Scientist’s job. But we should know it. Could be the responsible of a devops guy perhaps

Question 12

Name a couple of Big Data processing frameworks

Answer 12

• Hadoop MapReduce
• Apache Spark
• Apache Beam

Question 13

How does Hadoop MapReduce work? (I highly doubt this will be very relevant - See that chart in Notion)

Answer 13

A MR job splits data into independent chunks which are processed in-parallel by map tasks. Sorted map outputs are fed to reduce tasks.

YARN (Yet Another Resource Negotiator) offers resource management and job scheduling. It is a cluster management technology that became a core part of Hadoop 2.0

Question 14

What are the advantages of Apache Spark?

Answer 14

Apache Spark offers:

• In memory processing capability where data is cached in memory for rapid read and write
• Supports Data Analytics, machine learning algorithms
• Multiple languages (Such as Scala, Java, Python, R) support)

Question 15

What are the five components of Apache Spark?

Answer 15

• Apache Spark Core: Execution engine for spark (Provides in-memory computing)
• Spark SQL: Introduces data abstraction for semi-/structured data. Resilient Distributed Datasets (RDD) is a fundamental data structure of spark
• Spark Streaming: performs streaming analytics. Crunches data in mini-batches and performs RDD(Resilient Distributed Datasets) transformations on those mini-batches of data
• MLLib (Machine Learning Library): is a distributed machine learning framework
• GraphX: A distributed graph-processing framework

Question 16

How is Apache Beam different from Apache Spark?

Answer 16

• Apache Beam is a unified Batch+strEAM processing engine.
• Primarily focused on programming model

Beam does not contain infrastructure for distributed processing

Dunno, don’t focus too much on this, I think

Question 17

What are the primary components of Apache Beam?

Answer 17

Beam supports (What/Where/When/How) via Pipelines, PCollections, Transformations and Runners

• Pipelines(collection of processes) are streaming and processing user logic (Kinda similar to machine learning pipelines)
• PCollection is a distributed data set, processed inside pipeline
• Transform is a data processing operation/a step of a pipeline
• Runner used to execute pipelines. Spark runner for Apache Spark

Question 18

What are the core features of TensorFlow?

Answer 18

• Key ideas: Express a numeric computation as a graph
• Graph Nodes are operations with any number of inputs and outputs
• Graph edges are tensors which flow between nodes
• Computation graphs can be exported to a portable format, train here - run there
• TF implements autodiff and provides optimizers
• Minimize all sorts of loss functions

Also has a lot of data processing features: tf.keras, data loading and preprocessing (tf.data, tf.io), image processing (tf.image), signal processing (tf.signal)

Question 19

What is TensorFlow?

Answer 19

Its core is very similar to Numpy, but with GPU support

TensorFlow’s API revolves around tensors which flow from operation to operation hence name TensorFlow

Tensor is like a numpy ndarray

• Can create a tensor from Numpy array and vice versa
  
  • Can apply TF operations to Numpy arrays and numpy operations to tensors
• Ideally a multi-dimensional array, but can hold a scalar
• Helps during custom cost functions, custom metrics, custom layers

Note: Type conversions can significantly hurt performance

Question 20

Describe (in very broad terms) TensorFlow’s architecture

Answer 20

ARCHITECTURE:
High level (Python code) -> Keras/Data API -> Low-level python API/C++ -> Local/distributed execution engine -> CPU-/GPU-/TPU Kernels

Execution steps:

• Build a graph using variables and placeholders
• Deploy the graph for execution
• Train model by defining loss function and gradients computations

Question 21

True or False: Deep Learning frameworks hide mathematics and focus on design of neural nets

Answer 21

True

Question 22

Name some Deep Learning Frameworks

Answer 22

1. TensorFlow
2. Caffe
3. CNTK
4. PyTorch

Question 23

How do you train a Neural Network?

Answer 23

1) Build a computational graph from network definition
2) Input Training data and compute loss function
3) Update parameters

Define-and-run: DL Frameworks complete step one in advance (TensorFlow, Caffe)

Define-by-run: Combines steps one and two into a single step (PyTorch)
- Computational graph is not given before training, but obtained while training

Question 24

Not important, but pretty cool: ONNX (https://onnx.ai)

Acronym for: Open Neural Network eXchange

Answer 24

https://onnx.ai

Question 25

What is the difference between Distributed ML and non-distributed ML?

Answer 25

Non-distributed ML is what we've been doing throughout the course. Working on the whole dataset Distributed ML is a way of partitioning a dataset and working on it simultaneously. Implemented by Hadoop/Spark etc

Question 26

What is data parallelism?

Answer 26

A number of machines loads an identical copy of a DL model -> Training data is split into non-overlapping chunks and then fed to the workers Basically a way of splitting data and letting different "workers" to work on it simultaneously

Question 27

What is model parallelism?

Answer 27

Using the same data, but splitting the model into chunks and letting different "workers work on it simultaneously"

Question 28

Why were Recurrent Neural Networks made? (RNN)

Answer 28

RNNs were developed to solve learning problems (time series or sequential tasks) where information about past (i.e., past instances/events) is directly linked to making future predictions

Question 29

A recurrent neuron is different from a traditional neuron in that it...

Answer 29

maintains a memory or a state from past computations

Question 30

Name some different use cases for RNNs

Answer 30

Each of these have different graphs, showing the input -> memory -> output relationship. See Notion: - Image captioning - Sentiment analysis - Video classification - Machine translation and speech recognition

Question 31

How do you train RNNs?

Answer 31

Backpropagation through time (BPTT) 1. Unroll recurrent neuron across time instants 2. Apply backpropagation to unrolled neurons at each time layer same way it is done for traditional feedforward NN

Question 32

The challenge of training RNN is...

Answer 32

a vanishing and exploding gradient problem -> Solved by creating a new model called the Long Short-Term memory/LSTM