From the course: Spring: Spring Batch

Chunk-oriented processing

From the course: Spring: Spring Batch

Start my 1-month free trial

Chunk-oriented processing

“

- [Instructor] Up until this point, every step we have configured has been a tasklet. We'll now start to explore chunk-based steps, which are another type of step found within Spring Batch. Chunk-based steps use three components to complete their processing. These components are an ItemReader, an ItemWriter, and optionally, an ItemProcessor. The generic logic of chunk-based processing is to read items from a data store using an ItemReader, transform the items using the ItemProcessor, and then we write chunks of the data to another data store within a transaction using the ItemWriter. When reading, processing, and writing the items, we perform these operations on subsets of the data referred to as chunks. Our step will continue reading, processing, and writing chunks until the items in the data store are exhausted. Let's take a closer look at how this works. It is important to understand how data is processed through the a chunk-based step. When performing a chunk-based step, we typically provide a chunk size which determines how many items will be found within a chunk. In this example, our chunk size has been set to two, so you'll notice that each of the chunks contains two items. When processing starts, the ItemReader will read the first item in the chunk and then pass it to the processor for processing. It then repeats this process for the next item within the chunk. Once we've met the chunk size, the entire chunk will be passed to the ItemWriter and then will be written to a data store within a transaction. Once the chunk has been written, this process will repeat itself, will read the first item in the chunk, then process it, then read the second item the chunk and process that one, and then we're going to take the entire chunk and use the ItemWriter to write it to a data store. We'll see how to configure jobs for chunk-based processing over the next few lessons, focusing specifically on the ItemReader implementations that read items from a data store.

Contents