18-GPU-Atomic-Operations

Question 1

What is an atomic operation?

Answer 1

Operation that completes without interruption - appears instantaneous

Question 2

What is the load-compute-store problem?

Answer 2

Multiple threads reading, modifying, writing same variable cause races

Question 3

What is the histogram example?

Answer 3

Counting frequency of values - multiple threads incrementing same counters

Question 4

What is the problem with naive histogram?

Answer 4

Race conditions cause lost increments

Question 5

What is the solution?

Answer 5

Use atomic_inc() for thread-safe increments

Question 6

What are OpenCL atomic operations?

Answer 6

atomic_inc, atomic_dec, atomic_add, atomic_min, atomic_max, etc.

Question 7

What is atomic_inc(p)?

Answer 7

Atomically increments *p, returns old value

Question 8

What is atomic_cmpxchg(p

Answer 8

cmp

Question 9

What is the spinlock pattern?

Answer 9

while(atomic_cmpxchg(&lock, 0, 1) == 1) - busy wait for lock

Question 10

What is the problem with spinlocks on GPU?

Answer 10

Global memory lock causes contention, divergence issues

Question 11

What is lock-free programming?

Answer 11

Thread-safe data structures without locks, using atomics

Question 12

What is the linked list prepend problem?

Answer 12

Multiple threads adding to list head simultaneously

Question 13

What is the lock-free solution?

Answer 13

Use atomic_cmpxchg in loop to update head pointer atomically

Question 14

What is optimistic concurrency?

Answer 14

Assume operation succeeds, retry if conflict detected

Question 15

What is the trade-off with atomics?

Answer 15

Lower overhead than locks but limited to simple operations

Question 16

When to use atomics vs locks?

Answer 16

Atomics for simple operations, locks for complex multi-statement sections

Question 17

What is the histogram optimization?

Answer 17

Use local memory per work group, then atomic add to global

Question 18

Why use local memory for histogram?

Answer 18

Reduces contention on global memory locations

Question 19

What is the local histogram pattern?

Answer 19

Each work group builds local histogram, then atomically adds to global

Question 20

What is the performance benefit?

Answer 20

Less contention, local memory is faster

Question 21

What is the key insight about GPU atomics?

Answer 21

Essential for coordinating access to shared resources like histograms