From the course: Stream Processing Design Patterns with Spark

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Leaderboards: Use case design

Leaderboards: Use case design - Apache Spark Tutorial

From the course: Stream Processing Design Patterns with Spark

Start my 1-month free trial

Leaderboards: Use case design

- [Instructor] In this video, I will discuss the example use case for which we will build a leaderboard in this chapter. I will also explore the design for this example. The example use case for leaderboards is to build a gaming leaderboard for players. It shows the players who have the current top scores. The scores are continuously updated, and the score increments are published to Kafka. So the input is a Kafka topic that contains score increments by players. It's a key value input where the key represents the player. The value represents the score increment for the player. The goal is to maintain a player leaderboard based on the current total scores. What is the pipeline design for this use case? The input is a Kafka topic called streaming.leaderboards.input. Each message received would be a key value path. This needs to be converted to a data frame for further processing. The next stage is to update the leaderboard…

Contents