From the course: Apache Kafka Essential Training: Building Scalable Applications (2021)

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Kafka scaling and resiliency

Kafka scaling and resiliency

From the course: Apache Kafka Essential Training: Building Scalable Applications (2021)

Start my 1-month free trial

Kafka scaling and resiliency

- Two of the key features that make Kafka an ideal messaging system for big data are scalability and resiliency. Let's quickly review some of these capabilities. We will explore more about them throughout the course. Kafka is a horizontally scalable messaging system. It can process millions of messages per second and can handle terabytes of data. It is a system built for scalability. What are some of the key scalability features in Kafka? Messages can be spread across multiple topics and partitions. These can be stored and processed in parallel. Kafka brokers can be wired together to create clusters. Brokers in a cluster share the workload of managing and serving messages. Kafka supports multiple concurrent producers and consumers to enable parallel publishing and consumption of data. It also has asynchronous publishing and batching options to reduce latency, as well as network traffic. Consumer groups is another…

Contents