From the course: Apache Flink: Exploratory Data Analytics with SQL

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Challenges with streaming SQL

Challenges with streaming SQL - SQL Tutorial

From the course: Apache Flink: Exploratory Data Analytics with SQL

Start my 1-month free trial

Challenges with streaming SQL

- [Instructor] Flink provides a unified analytics platform for both batch and streaming data. However, there are some unique characteristics and limitations when analyzing streaming data. Let's try to understand what they are and how they impact data analytics. Streaming data is unbounded, at any point during the analysis, it's not possible to know the total number of records we are dealing with. New data keeps coming in at mostly unpredictable intervals. Constant new data also means that the results are not repeatable. The same query executed five minutes back will generate different results if executed again. This also means that the results derived from stream queries expire quickly and they need to be executed again to get the latest status. Most stream processing applications deal with state and they became a key part in the analytics too. For example, we want may want to analyze current active users on a website…

Contents