Module 1

Question 1

_ involves processing instructions one at a time, using only a single processor, without distributing tasks across multiple processors.

Answer 1

Serial computing

or sequential computing

Question 2

_ was introduced as computer science evolved to address the slow speeds of serial computing.

Answer 2

Parallel computing

Question 3

_ is a method where parallel programming enables computers to run processes and perform calculations simultaneously.

Answer 3

Parallel processing

Question 4

_ is a process where large computing problems are broken down into smaller problems that multiple processors can solve simultaneously.

Answer 4

Parallel computing

Also known as parallel programming

Multiple processors working simultaneously on different parts of a task.

Example: New British weather supercomputer MetOffice’s

Question 5

Real-world applications of parallel computing span diverse domains, from scientific simulations to big data analytics and high-performance computing.

Answer 5

Noted

Question 6

Parallel computing architectures enable efficient processing and analysis of large datasets, sophisticated simulations, and complex computational tasks.

Answer 6

• Task Distribution: Parallel computing is the process by which a supercomputer can split the whole grid into sub-grids
• Simultaneous Computation: Thousands of processors work simultaneously on different parts of the grid to calculate the data which is stored at different locations
• Communication between Processors: The main reason for processors to communicate with each other is the fact that the weather for one part of the grid can have an impact on the areas adjacent to it

Question 7

_ consist of multiple processing units, or ‘cores,’ on a single integrated circuit (IC). This structure facilitates parallel computing, which enhances performance while potentially reducing power consumption.

Answer 7

Multicore processors

The need for higher performance, faster response times, increased functionality, and energy efficiency has never been more pressing.

Multi-cores, a system can perform multiple tasks at once.

Question 8

Parallel Computing Benefits

Answer 8

1. SPEED AND EFFICIENCY: allows tasks to be completed faster by dividing them into smaller sub-tasks that can be processed simultaneously by multiple processors or cores.
2. HANDLING LARGE DATA SETS (Scalability): essential for processing large data sets that would be impractical or too slow to handle sequentially
3. SOLVING COMPLEX PROBLEMS: allows for the tackling of such problems by leveraging multiple processors.
4. FAULT TOLERANCE: Parallel systems can be designed to be fault-tolerant, meaning they can continue to operate even if one or more processors fail. This improves the reliability and availability of the system.

Question 9

PARALLEL COMPUTING IS A VERSATILE TOOL APPLIED IN MANY DIFFERENT AREAS OF INDUSTRY, INCLUDING:

Answer 9

1. SCIENTIFIC SIMULATIONS: Parallel computing is required for simulations that are complex and belong to such fields as physics, chemistry, and biology. (It enables researchers to model large-scale systems)
2. DATA ANALYSIS: In genomics, astronomy, and finance, parallel computing is necessary for the analysis of large data sets. (massive datasets faster processing of data, enabling researchers to extract valuable insights and make informed decisions)
3. MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE: Setting up a big machine learning model like a neural network needs a huge computational (accelerates the training process, enabling the development of more advanced AI systems.)

Question 10

Parallel computing plays a vital role in addressing complex problems and enabling advancements in various fields. It provides the computational power necessary for scientific research, data analysis, machine learning, high-performance computing, and other demanding applications

Answer 10

Noted

Question 11

Named after the Hungarian
mathematician John von Neumann

Answer 11

Von Newmann Architecture

Question 12

Von Newmann Architecture was named after the Hungarian mathematician _

Answer 12

John von Neumann

Question 13

A _ computer uses the
stored-program concept

Answer 13

von Neumann

The CPU executes a stored program that specifically a sequence of read and write operations on the memory.

Question 14

The _ gets the instructions and/ or data from the memory, decodes the instructions, and then sequentially performs them.

Answer 14

CPU

Question 15

One of the more widely used
classifications, in use since 1966.

Answer 15

Flynn’s Classical Taxonomy

Question 16

Distinguishes multi-processor computer architectures according to how they can be classified along two independent dimensions of instruction and data.

Answer 16

Flynn’s Classical Taxonomy

Question 17

According to Flynn’s Classical Taxonomy, each dimension can have only one of two possible states: _ or _.

Answer 17

single or multiple

Question 18

In Flynn’s Matrix Array, the matrix defines four classification:

Answer 18

SISD | SIMD
MISD | MIMD

Y = INSTRUCTION

X = DATA, Y = INSTRUCTION

SISD - Single Instruction, Single Data
SIMD - Single Instruction, Multiple Data
MISD - Multiple Instruction, Single Data
MIMD - Multiple Instruction, Multiple Data

Question 19

• A single processor takes data from the single address memory and performs a single instruction on the data at the time.

Answer 19

Flynn’s SISD

Pipelining can be implemented, but only one instruction will be executed at a time.

Question 20

A single instruction is executed on a multiple different pieces of data.

Answer 20

Flynn’s SIMD

Instructions can be performed sequentially taking advantage of pipelining or in parallel using a multiple processors.

GPU, containing vector processors and array processors, are commonly SIMD system.

Question 21

Multiple processors work on the same data performing different instructions at the same time.

Answer 21

Flynn’s MISD

Example: Space shuttle flight control system

Question 22

Autonomous processors perform operations on differences pieces of data either independently or as part of shared memory

Answer 22

Flynn’s MIMD

Several different instruction can be executed at the same time using different streams

Question 23

Main reasons for using parallel programming

Answer 23

1. Save time and/or money
2. Solve larger / more complex problems
3. Provide concurrency
4. Complex, large datasets

Question 24

The word “distributed” in distributed computing is similar to terms such as?

Answer 24

distributed system
distributed programming
distributed algorithm

Question 25

Originally referred to independent computers interconnected via a network, that is capable of collaborating on a task.

Answer 25

Distributed Computing

Question 26

Networks of interconnected computers that work together to achieve a common goal.

Answer 26

Distributed Computing

Question 27

In distributed computing, computers are often spread across different locations and connected through a _, such as the internet or a local area network (LAN).

Answer 27

network

Question 28

In a distributed computing system, the workload is divided among the various _, which communicate and coordinate their efforts to achieve a common goal.

Answer 28

nodes

Question 29

Computers can be physically close ( _ ) or far apart ( _ ).

Answer 29

local network wide area network

Question 30

Benefits of distributed computing

Answer 30

1. Performance Improvement 2. Scalability 3. Resilience and redundancy 4. Cost-effectiveness 5. Efficiency Distributed applications 6. Geographical Distribution 7. Resource Sharing

Question 31

Advantages of Distributed Computing

Answer 31

1. **Leverage Commodity Hardware:** Use less expensive, off-the-shelf hardware instead of costly, specialized servers. 2. **Horizontal Scaling:** Easily add more nodes (computers) to a distributed system to handle increased workloads. 3. **Fault Tolerance:** if one node fails, the system can continue to operate, ensuring uninterrupted service. 4. **Reliability:** Can recover quickly from node failures or network issue

Question 32

Disadvantages of Distributed Computing

Answer 32

1. **Network Latency:** **Increased Complexity:** Managing network latency can be complex, requiring careful consideration of factors like network topology, bandwidth, and routing protocols. 2. **Coordination Overhead:** Ensuring that multiple nodes coordinate their actions can be challenging, especially in distributed systems with many components. 3. **Security Concerns:** **Vulnerabilities:** more susceptible to security threats like hacking, data breaches, and denial-of-service attacks due to their interconnected nature. 4. **Debugging and Troubleshooting:** **Complexity:** Identifying and resolving issues in distributed systems can be challenging due to their distributed nature and the potential for interactions between multiple components.

Question 33

Example of Servers of Distributed Computing

Answer 33

1. **Cloud Computing:** Services like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform rely on distributed computing to offer scalable and reliable cloud services. 2. **Artificial Intelligence and Machine Learning:** Artificial Intelligence (AI) and Machine Learning (ML) are two of the most exciting and rapidly developing fields in technology today. They are also among the most notable use cases for distributed computing. 3. **Scientific Research and High-Performance Computing (HPC):** where distributed is used extensively in these fields, distributed computing is used to solve complex scientific problems that require enormous computational resources.

Question 34

# Distributed Computing Concept A _ in a computer network is a device that is connected to other devices within the same computer network.

Answer 34

node ## Footnote It can be a computer, a server, a switch, a hub, a router, or any other device that has an IP address and can communicate with other devices over a network.

Question 35

# Distributed Computing Concept A node can be from _ or _.

Answer 35

open group close network

Question 36

# Distributed Computing Concept Resources can be anything, such as?

Answer 36

On file On services Storage facility On network

Question 37

# Distributed Computing Concept _ refers to hiding the complexities of the system’s implementation details from users and applications.

Answer 37

Transparency ## Footnote It aims to provide a seamless and consistent user experience regardless of the system’s underlying architecture, distribution, or configuration. Keep as much as possible hidden from users.

Question 38

# Distributed Computing Concept A logical layer on top of nodes collectively.

Answer 38

Middleware

Question 39

Provides communication and security services while handling failure and other complexities in distributed systems

Answer 39

Middleware | the backbone of the distributed system

Question 40

_ smooth collaboration between operations and events.

Answer 40

Coordination

Question 41

_ order events, control access to resources

Answer 41

Synchronization

Question 42

_ helps with better management of complexities and describes how nodes communicate and interact in a system

Answer 42

Architectural Model

Question 43

3-Sub Paced of Distributed Computing

Answer 43

1. Cluster Computing 2. Cloud Computing 3. Grid Computing

Question 44

_ breaks down the problem across several networked computing device

Answer 44

Distributed computing

Question 45

_ is the world’s largest coding community for children and a coding language with a simple visual interface design to gentle introduction of programming concepts.

Answer 45

Scratch

Question 46

_ is an increasingly popular choice as a first programming language in computer science curricula.

Answer 46

Python

Question 47

These architectures use smart clients that contact a server for data, and then format and display that data to the user.

Answer 47

Client-server architectures

Question 48

Typically used in application servers, these architectures use web applications to forward requests to other enterprise services.

Answer 48

N-tier system architectures

Question 49

These architectures divide all responsibilities among all peer computers, which can serve as clients or servers.

Answer 49

Peer-to-peer architectures

Question 50

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING NUMBER OF COMPUTER SYSTEMS INVOLVED

Answer 50

**Parallel Computing:** A task is divided into multiple sub-tasks which are then allotted to different processors on the same computer system. **Distributed Computing:** A number of unified computers work towards a common task while communicating with each other with the help of message passing.

Question 51

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING DEPENDENCY BETWEEN PROCESSES

Answer 51

**Parallel Computing:** A single physical computer system hosts multiple processors. **Distributed Computing:** Multiple physical computer systems are present in the computer systems.

Question 52

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING SCALABILITY

Answer 52

**Parallel Computing:** The systems that implement parallel computing have limited scalability. **Distributed Computing:** Easily scalable as there are no limitations on how many systems can be added to a network.

Question 53

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING RESOURCE SHARING

Answer 53

**Parallel Computing:** All processors share the same memory. **Distributed Computing:** Computers have their own memory and processors.

Question 54

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING SYNCHRONIZATION

Answer 54

**Parallel Computing:** All processors use the same master clock for synchronization. **Distributed Computing:** Networks have to implement synchronization algorithms.

Question 55

# PARALLEL COMPUTING VS DISTRIBUTED COMPUTING USAGE

Answer 55

**Parallel Computing:** Generally preferred in places requiring faster speed and better performance. **Distributed Computing:** Generally preferred in places requiring high scalability.

Question 56

**Parallel computing** uses multiple processors within a single computer system to divide and process tasks simultaneously, with shared memory and synchronization through a master clock. It is ideal for faster performance but has limited scalability. In contrast, **distributed computing** involves multiple independent systems working together via message passing, each with its own memory and processors. This allows for greater scalability and is suited for tasks requiring extensive resource sharing across systems.

Answer 56

Noted