2 - Hardware & Architectures

Question 1

What is optimised for low-latency communications?

Answer 1

CPU

Question 2

What is optimised for data parallel and high throughout computations

Answer 2

GPU

Question 3

Which has the larger cache? CPU or GPU?

Answer 3

CPU

Question 4

What has more transistors dedicated to computation? CPU or GPU?

Answer 4

GPU

Question 5

What is better for real-time applications? CPU or GPU?

Answer 5

CPU

Question 6

Discuss the CPU

Answer 6

• Optimised for law latency computations
• Large caches
• Fewer ALUs
• Good for real-time applications

Question 7

Discuss the GPU

Answer 7

• Optimised for data-parallel and high throughput computations
• Smaller caches
• More transistors dedicated to computation
• Good if enough work to hide latency

Question 8

What are 4 pieces of parallel hardware?

Answer 8

• Pipelines, vector instructions in CPU
• Multi-core CPUs and multi-processors
• Dedicated parallel processing units
• Distributed systems

Question 9

Name 4 applications of parallel computing

Answer 9

Autonomous Cars, Augmented Reality, Video Games and Weather Forecasting

Question 10

Why is parallel programming less intuitive than serial programming?

Answer 10

Non-trivial communication and synchronisation

Question 11

Why does adding processors not always improve performance of serial programs?

Answer 11

The serial programs are unaware of the existence of multiple processors

Question 12

What was single processing power performance driven by?

Answer 12

The increasing density of transistors

Question 13

What are parallel programs?

Answer 13

Programs that exploit the power of multiple processors

Question 14

What is a shared memory parallel system?

Answer 14

Where processors operate independently but share the same memory

Question 15

What is a distributed memory system?

Answer 15

Where the processors operate independently but have their own local memory

Question 16

What is pipelining?

Answer 16

Where function units are arranged in stages

Question 17

What do we need to think about when writing parallel programs?

Answer 17

Communication among cores, load balancing and synchronisation of cores

Question 18

Why can’t we translate serial programs into parallel programs?

Answer 18

Translations cannot deal with the fundamental shift in algorithmic structure required for effective parallelisation

Question 19

What is a parallel pattern?

Answer 19

Valuable algorithmic structure commonly seen in efficient parallel programs

Question 20

What is serialisation?

Answer 20

Act of putting some set of operations into a specific order

Question 21

What is locality?

Answer 21

Memory accesses close together in time and span are cheaper than far apart ons

Question 22

What is the difference between implicit and explicit parallelism?

Answer 22

The programmer has no access to implicit parallelism, but they do have access to explicit parallelism

Question 23

Name a type of implicit parallelism

Answer 23

Pipelines and vector instructions in CPU

Question 24

Name 3 types of explicit parallelism

Answer 24

Multi-core CPUS and multi-processors, dedicated parallel processing units, distributed systems

Question 25

What is Flynn's Taxonomy?

Answer 25

Classifies multi-processor computer architectures according to two independent dimensions, instruction streams and data streams

Question 26

What are the two possible states of dimensions within Flynn's Taxonomy?

Answer 26

Single and multiple

Question 27

What are the four possible classifications within Flynn's Taxonomy?

Answer 27

SISD, SIMD, MISD, MIMD

Question 28

What is SISD?

Answer 28

Single instruction, single data

Question 29

What is SIMD?

Answer 29

Single instruction, multiple data

Question 30

What is MISD?

Answer 30

Multiple instruction, single data

Question 31

What is MIMD?

Answer 31

Multiple instruction, multiple data

Question 32

Describe SISD

Answer 32

A serial computer, only one instruction stream is being acted on by the CPU during one clock cycle and only one data stream is being used as input during one clock cycle

Question 33

What is SISD good for?

Answer 33

Traditional single processor/single core CPUS and real-time applications

Question 34

Describe SIMD

Answer 34

Parallel, all processing units execute the same instruction at a clock cycle, each processing unit can operate on a different data element

Question 35

What is SIMD good for?

Answer 35

Modern CPUs and GPUs, best for problems with a high degree of regularity e.g image processing

Question 36

Describe MIMD

Answer 36

Most common type of parallel computer, every processor may be executing a different instruction stream, every processor may be working with a different data stream

Question 37

What is MIMD good for?

Answer 37

Multi-core CPUs, computing clusters and grids

Question 38

What is worth noting about MIMD?

Answer 38

Many MIMD architectures also include SIMD execution sub-components

Question 39

Describe MISD

Answer 39

Each processing unit operates on the data independently via separate instruction streams, a single data stream is def into multiple processing units

Question 40

What is MISD good for?

Answer 40

CryptographY

Question 41

Which of Flynn's Taxonomies is rare and uncommon?

Answer 41

MISD

Question 42

What are the two memory architectures for SIMD and MIMD?

Answer 42

Shared and distributed

Question 43

What is a shared memory architecture?

Answer 43

Processors operate independently but share the same memory - global address space, changes in memory by one processor are visible to all other processors

Question 44

What is distributed memory?

Answer 44

Processors operate independently but have their own local memory, memory addresses in one processor do not map to another processor - no global address space

Question 45

What are the pros of the shared memory architecture?

Answer 45

Global address space is easy to use and program, and data sharing between tasks is fast due to the proximity of memory to CPUs

Question 46

What are the cons for the shared memory architecture?

Answer 46

Adding more CPUs can increase traffic on shared memory-CPU path and the programmer is responsible for synchronisation to ensure correct access to global memory

Question 47

What are the pros to the distributed memory architectures?

Answer 47

Each processor can rapidly access its own memory without interference, and memory is scalable (increase the number of processors and size of memory increased)

Question 48

What are the cons to the distributed memory architecture?

Answer 48

The programmer is responsible for data communication and non-uniform memory access times

Question 49

What is the GPU?

Answer 49

Designed for manipulating computer graphics and image processing, highly parallel, more efficient than CPUs when processing large blocks of visual data in parallel

Question 50

What are the different realisations of the GPU?

Answer 50

Dedicated expansion video card, integrated into the CPU, embedded on motherboard

Question 51

What is GPGPU?

Answer 51

General-purpose computing on GPU. Application of GPU for applications other than graphics, with large datasets and complex computations

Question 52

When might you use GPGPU?

Answer 52

Physics simulation, AI, weather forecasting

Question 53

What is latency?

Answer 53

Time to solution. Minimises time at the expensive of power

Question 54

What is throughout?

Answer 54

Quantity of tasks processed per unit of time, minimises energy per operation

Question 55

What is heterogeneous computing?

Answer 55

Combining both a latency processor (CPU) with a throughput processor (GPU)