From the course: Apache Kafka Essential Training: Getting Started (2021)
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Consumer offset management
From the course: Apache Kafka Essential Training: Getting Started (2021)
Consumer offset management
- [Narrator] When Kafka pushes messages to consumers it also needs to ensure reliable delivery. Kafka ensures at least once guaranteed delivery using consumer offsets. What is a consumer offset? It is a number to track message consumption by each consumer and partition. As each message is received by Kafka, it allocates a message ID to the message. Kafka then maintains the message ID offset on a by consumer and by partition basis to track consumption. Kafka brokers keep track of both what is sent to the consumer and what is acknowledged by the consumer by using two offset values. The Current offset value tracks the last message ID that was sent to the consumer. The committed offset value tracks the last message that is acknowledged by the consumer. As consumers receive a message, they have the option of acknowledging immediately or after making sure that all required processing is done. This helps consumers to manage…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.