BlockingQueue

Producer - consumer problem arises in multi thread environment. This can be defined when there are one or more threads generate data independently and same time one or many threads read those data to process independently.

To solve this we can have an array and there would be synchronized method to access. So when one thread only can either read/write at same time. If the array is full, producer threads need to wait. And if the array is empty consumer threads need to wait.

java.util.concurrent package comes with Blocking queue which is very handy to solve this problem.

Blocking queue is an interface. ArrayBlockingQueue  is an concrete implementation of interface. It allow both wait or fail for producer if the queue is full, same way for consumer it fails or waits if the queue is empty.

Some sample codes can be foundin https://github.com/pallabrath/myexpjava/tree/master/myexpjava/src/demo/blockingQueue

What we discussed here is in the scope of single instance. If we want scale then there would be more than one instance. In that case of distributed environment we need different kind of solution. Java message queue fit into this situation.

Again if we scale further and we want to process data in large volume, Kafka goes well in the distributed environment. Kafka itself run on one or many node to scale.

We will discuss Kafka in a separate thread.