What is a Kafka Producer?
A Producer is a client application that publishes (writes) records to Kafka topics. Producers decide which partition to send data to and can configure acknowledgment settings for reliability.
What is a Kafka Consumer?
A Consumer is a client application that reads records from Kafka topics. Consumers subscribe to topics and process the stream of records produced to them, tracking their position via offsets.
What is a Consumer Group?
A Consumer Group is a set of consumers that work together to consume topics. Partitions are distributed among consumers in the group, providing both scalability and fault tolerance.
How does Kafka ensure message ordering?
Kafka guarantees order only within a partition, not across partitions. For strict ordering, use single-partition topics or use keys to ensure related messages go to the same partition.
What is the role of message keys in partitioning?
When a producer sends a message with a key, Kafka hashes the key to determine the target partition. This ensures all messages with the same key go to the same partition, maintaining order for related messages.
What are Producer Acknowledgments (acks)?
acks=0: Fire and forget, acks=1: Wait for leader acknowledgment, acks=all: Wait for all in-sync replicas acknowledgment. Higher acks increase durability but decrease throughput.
What is Consumer Offset?
Offset is a sequential ID number assigned to each record in a partition. Consumers track their current position (offset) in each partition, allowing them to resume from where they left off.
What is the difference between Earliest and Latest offset reset?
Earliest: Start from beginning of partition. Latest: Start from only new messages. This setting is used when a consumer group starts for the first time or has invalid offsets.
