Component Roles

Question 1

What are the three main responsibility planes in Kafka?

Answer 1

1. Control plane: cluster metadata and leadership management, handled by the controller. 2. Data plane: partition reads, writes, and replication, handled by partition leaders and followers. 3. Consumer-group plane: membership, rebalancing, and offset commits, handled by the group coordinator.

Question 2

What does the controller do in Kafka?

Answer 2

The controller manages cluster metadata and control-plane decisions such as topic and partition metadata, broker membership changes, and partition leader elections. It is not in the normal produce/fetch data path.

Question 3

What does a partition leader do?

Answer 3

A partition leader is the authoritative broker for one partition. Producers write to it, consumers fetch from it, and followers replicate from it.

Question 4

What does a partition follower do?

Answer 4

A follower is a non-leader replica that actively fetches data from the leader to stay caught up and can become leader if elected.

Question 5

Is Kafka replication push-based or pull-based?

Answer 5

Pull-based. Followers send fetch requests to the leader and copy data from it.

Question 6

What is ISR in Kafka?

Answer 6

ISR stands for In-Sync Replicas. It is the set of replicas sufficiently caught up with the leader and eligible for safe leader election.

Question 7

Is every follower automatically in the ISR?

Answer 7

No. A follower is any non-leader replica. It is only in the ISR if it is sufficiently caught up.

Question 8

Does replication factor include the leader?

Answer 8

Yes. Replication factor is the total number of replicas for a partition, including the leader. RF=3 means 1 leader and 2 followers.

Question 9

What is the difference between replication factor and ISR size?

Answer 9

Replication factor is the configured total number of replicas. ISR size is the number of replicas currently in sync. RF is static; ISR is dynamic.

Question 10

What does acks=all mean?

Answer 10

The leader waits for all replicas currently in the ISR to acknowledge the write, not all assigned replicas in the replication factor.

Question 11

What does min.insync.replicas do?

Answer 11

It defines the minimum ISR size required for a write with acks=all to succeed. If ISR is below that threshold, the write fails.

Question 12

Why is acks=all often misunderstood?

Answer 12

Because many think it means all assigned replicas. In reality, it means all replicas currently in the ISR.

Question 13

What does the group coordinator do?

Answer 13

The group coordinator manages consumer-group state: membership, joins, heartbeats, rebalances, and offset commits.

Question 14

Where are consumer offsets stored?

Answer 14

In the internal compacted Kafka topic \_\_consumer_offsets.

Question 15

Who handles an offset commit?

Answer 15

The consumer sends the offset commit to the group coordinator, which persists it in \_\_consumer_offsets.

Question 16

Why are offset commits handled by the group coordinator and not the partition leader?

Answer 16

Because committed offsets are consumer-group progress state, not part of the business topic’s data stream.

Question 17

Can a broker have multiple roles at the same time?

Answer 17

Yes. A broker can be leader for some partitions, follower for others, coordinator for some consumer groups, and in combined KRaft mode also have the controller role.

Question 18

Are Kafka roles global or scoped?

Answer 18

Scoped. A broker is leader for specific partitions, coordinator for specific groups, and controller only if configured and currently active in the controller quorum.

Question 19

What is the difference between controller and partition leader?

Answer 19

The controller manages cluster metadata and leader elections. The partition leader handles actual reads, writes, and replication for one partition.

Question 20

What is the difference between controller and group coordinator?

Answer 20

The controller manages cluster-level metadata and leadership. The group coordinator manages consumer-group membership, rebalances, and offset commits.

Question 21

What is the difference between partition leader and group coordinator?

Answer 21

A partition leader serves actual topic data for one partition. A group coordinator manages group control functions such as join, heartbeat, rebalance, and commit.

Question 22

How does a producer find where to send records?

Answer 22

It first gets cluster metadata from a broker, learns which broker is leader for the target partition, then sends the record to that partition leader.

Question 23

Does a producer send records to the controller?

Answer 23

No. A producer sends records to the leader of the target partition.

Question 24

How does a consumer get actual records?

Answer 24

It fetches records from the leader of its assigned partitions.

Question 25

Does a consumer fetch records from the group coordinator?

Answer 25

No. It uses the group coordinator for membership and commits, but fetches actual records from partition leaders.

Question 26

What does a consumer use the group coordinator for?

Answer 26

Joining the group, participating in rebalance, sending heartbeats, and committing offsets.

Question 27

What is a bootstrap broker?

Answer 27

The initial broker a client contacts for metadata. It helps the client learn leaders, coordinators, and broker endpoints.

Question 28

Can admin requests go to any broker?

Answer 28

Many metadata-changing admin requests can go to a broker listener, and that broker can forward them to the active controller. This is not true for all Kafka traffic.

Question 29

Which Kafka traffic must go to specific brokers?

Answer 29

Produce and fetch requests must go to the correct partition leader. Consumer-group membership and offset-commit operations must go to the correct group coordinator.

Question 30

What is the classic consumer-group rebalance flow?

Answer 30

Find coordinator, join group, one consumer becomes group leader, assignment is computed, assignments are distributed via SyncGroup, consumers fetch from partition leaders, and heartbeats keep them in the group.

Question 31

In the classic group protocol, what is the group leader?

Answer 31

A temporary consumer-side role during rebalance. One consumer member helps compute partition assignments for the group.

Question 32

What is the difference between group coordinator and group leader?

Answer 32

The group coordinator is a broker-side role managing the group protocol. The group leader is a temporary consumer member chosen during classic rebalance to help compute assignments.

Question 33

What are the three different 'leader-like' concepts that are easy to confuse?

Answer 33

1. Active controller: metadata leader at cluster level. 2. Partition leader: data leader for one partition. 3. Group leader: temporary consumer member during classic rebalance.

Question 34

What usually triggers a rebalance?

Answer 34

A new consumer joining, a consumer leaving, a consumer timing out, or any change affecting partition ownership in the group.

Question 35

What are heartbeats for?

Answer 35

They tell the group coordinator that the consumer is still alive as a member of the group.

Question 36

What is `session.timeout.ms` about?

Answer 36

Liveness timeout. If heartbeats are not received within this time, the coordinator considers the consumer dead and triggers rebalance.

Question 37

What is `max.poll.interval.ms` about?

Answer 37

Processing-progress timeout. If the consumer does not call `poll()` again within this interval, Kafka considers it stuck or too slow and rebalances the group.

Question 38

What is the difference between heartbeat liveness and poll progress?

Answer 38

Heartbeats prove the consumer is alive to the coordinator. Poll progress proves it is still actively participating in the processing loop.

Question 39

Can a consumer be alive but still lose its assignment?

Answer 39

Yes. If it does not call `poll()` again before `max.poll.interval.ms`, rebalance can happen even if the process is still running.

Question 40

What happens to partition ownership during rebalance?

Answer 40

Previously owned partitions can be revoked from one consumer and reassigned to another.

Question 41

Why is revocation important for offset commits?

Answer 41

If a consumer loses a partition before committing its latest processed offset, the next consumer may resume from an older committed offset and reprocess messages.

Question 42

What does a committed offset represent?

Answer 42

The next offset to read, not the last record already processed.

Question 43

If a consumer committed offset 850, from which offset will the next consumer resume?

Answer 43

Offset 851.

Question 44

What is `commitSync()`?

Answer 44

A blocking offset commit. The consumer waits until the commit result is known.

Question 45

What is `commitAsync()`?

Answer 45

A non-blocking offset commit. The consumer continues without waiting synchronously for success.

Question 46

Why use `commitAsync()` during normal processing?

Answer 46

To avoid slowing the processing loop with blocking commits and improve throughput.

Question 47

What is the risk of `commitAsync()` around rebalance?

Answer 47

A partition may be revoked before the latest processed offsets were durably committed, so another consumer may resume from an older offset and reprocess messages.

Question 48

What is a common manual commit pattern?

Answer 48

Use `commitAsync()` during normal processing for throughput, and `commitSync()` in revocation or shutdown handling for the latest completed work.

Question 49

Why use `commitSync()` in `onPartitionsRevoked()`?

Answer 49

To durably save the latest fully processed offsets for partitions about to be lost, reducing duplicate processing after reassignment.

Question 50

What is the difference between last processed offset and last committed offset?

Answer 50

Last processed offset is what the application handled in memory. Last committed offset is what the group coordinator durably recorded as progress.

Question 51

Why can duplicate processing happen after rebalance or crash?

Answer 51

Because the next consumer resumes from the last committed offset, not from the last record processed only in memory.

Question 52

What is KRaft?

Answer 52

Kafka’s modern metadata architecture in which Kafka manages its own metadata using a Raft-based controller quorum instead of ZooKeeper.

Question 53

What is the difference between KRaft vs ZooKeeper and classic vs consumer protocol?

Answer 53

KRaft vs ZooKeeper is about cluster metadata architecture. Classic vs consumer protocol is about consumer-group rebalance protocol. They are separate concerns.

Question 54

Which consumer-group protocol should be prioritized for CCDAK study?

Answer 54

The classic consumer-group protocol.

Question 55

What election or selection mechanisms are important to know in Kafka?

Answer 55

Active controller election, partition leader election, and in the classic protocol the temporary selection of a consumer group leader during rebalance.

Question 56

How is a new partition leader chosen after failure?

Answer 56

From eligible in-sync replicas. Only ISR members are eligible for safe leader election.

Question 57

What is the preferred leader for a partition?

Answer 57

The first replica in the configured replica assignment list. Kafka may later move leadership back to it.

Question 58

Is the new partition leader always just the first ISR entry?

Answer 58

No. The safe rule is that the new leader is elected from eligible ISR replicas. The preferred leader is separately the first replica in the assignment list.

Question 59

What should be memorized about leader elections for the exam?

Answer 59

Know that controller leadership and partition leadership are different, partition leaders are elected from ISR, and the preferred leader is the first replica in the replica list.