From the course: Designing Highly Scalable and Highly Available SQL Databases

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Message queues to buffer ingested data

Message queues to buffer ingested data

From the course: Designing Highly Scalable and Highly Available SQL Databases

Start my 1-month free trial

Message queues to buffer ingested data

“

- [Instructor] Now, sometimes we need to use message queues when we're working with highly scalable and highly available services, and particularly database applications. The reason we often decouple services when we're in a distributed environment, it is because the services may function and ingest data and process data at different rates. So here's a really trivial example where we have two services. Service one generate some data or produces some data, which is then consumed by service two. Now, this may not be a problem at all if service one and service to basically generate and consume at the same rates, but service one may ingest or process data faster than service two can actually handle it, and that's why we need to buffer. So when we buffer, we use message queues. Now, typically message queues, first of all, are highly scalable, so that's a key thing for enabling highly scalable databases. And they're able to…

Contents