本文介绍了Java中如何实现生产者-消费者模式，包括使用synchronized、wait()、notifyAll()方法，以及如何安全地停止程序和通过管程优化。通过实例展示了生产者和消费者线程的交互，并提供了完整源代码链接。
Java实现生产者-消费者模式
1. 什么是生产者-消费者模式
生产者-消费者模式主要三种对象：生产者、消费者、队列。有一些生产者，每个生产者会生产一些产品，将它们放到队列中。这些产品会被消费者从队列中取出使用。队列存在最大长度，当队列满时，生产者需要等待消费者消费后才能继续将生产的产品放入队列中，当队列为空时，消费者需要等待生产者将新的产品放入队列才能继续消费产品。
2. 准备工作
我们可以把每个生产者和消费者看成一个线程，并且假设队列实现了如下的接口
队列接口
public interface IBlockedQueue <T> {
void enqueue(T item, Function<Integer, Void> onSuccess) throws InterruptedException;
T dequeue(Function<Integer, Void> onSuccess) throws InterruptedException;
int getTotalProduced();
int getTotalConsumed();
}
其中onSuccess函数接受队列当前的长度为参数，并且会在enqueue或dequeue操作成功之后马上被执行。这个函数帮助我们更容易地观察程序的运行情况。
然后就可以定义生产者和消费者
生产者
public static class Producer extends Thread {
public int id;
public IBlockedQueue<Product> queue;
public Producer(int id, IBlockedQueue<Product> queue) {
super("Producer-" + id);
this.id = id;
this.queue = queue;
}
@Override
public void run() {
try{
for (int i = 0; i < 100; i++) {
sleep(PRODUCE_TIME);
queue.enqueue(new Product(id * 1000 + i), size -> {
System.out.printf("Queue now has %d items\n", size);
return null;
});
System.out.printf("Producer %d produced item %d\n", id, id * 1000 + i);
}
} catch (InterruptedException e) {
this.interrupt();
}
}
}
消费者
public static class Consumer extends Thread {
public int id;
public IBlockedQueue<Product> queue;
private boolean isDone = false;
public Consumer(int id, IBlockedQueue<Product> queue) {
super("Consumer-" + id);
this.id = id;
this.queue = queue;
}
@Override
public void run() {
try{
while(true) {
sleep(CONSUME_TIME);
Product p = queue.dequeue(size -> {
System.out.printf("Queue now has %d items\n", size);
return null;
});
if (p == null) { // no more products
//这一行现在永远不会执行，之后会做相应改动
break;
}
System.out.printf("Consumer %d consumed item %d\n", id, p.id);
}
} catch (InterruptedException e) {
this.interrupt();
}
}
}
3. 通过synchronized方法, wait(), notifyAll()实现
这种方法等于使用this自带的锁来进行同步，具体办法是将入队和出队设成syncrhronized。生产者会在入队时（得到锁之后）检查队列是否为满，如果满了，就释放掉锁并进入阻塞（wait())。等到队列有了新的空位，消费者通过notifyAll()唤醒所有线程，此时被唤醒的生产者再次检查队列，发现了新的位置，就可以再继续将产品入队了，入队完后，生产者会用notifyAll()通知消费者去消费。相对的，消费者也会在出队时等待直至队列不为空，出队完通知。
public class SimpleBlockedQueue<T> implements IBlockedQueue<T> {
final int LENGTH = 10;
Queue<T> queue = new ArrayDeque<>();
int count = 0;
int totalProduced = 0;
int totalConsumed = 0;
public synchronized void enqueue(T item, Function<Integer, Void> onSuccess) throws InterruptedException {
while (queue.size() == LENGTH) {
this.wait(); //wait for consumer
}
queue.add(item);
this.notifyAll(); //notify consumers
totalProduced++;
if (onSuccess != null) {
onSuccess.apply(this.queue.size());
}
}
public synchronized T dequeue(Function<Integer, Void> onSuccess) throws InterruptedException {
while (queue.size() == 0 && ! c.isDone()) {
this.wait(); //wait for producer
}
if (queue.size() == 0 && c.isDone()) {
return null;
}
T item = queue.remove();
this.notifyAll(); //notify producers
totalConsumed++;
if (onSuccess != null) {
onSuccess.apply(this.queue.size());
}
return item;
}
public int getTotalProduced() {
return totalProduced;
}
public int getTotalConsumed() {
return totalConsumed;
}
}
4. 运行测试
public class Test {
static final int PRODUCE_TIME = 10;
static final int CONSUME_TIME = 20;
static final int PRODUCER_COUNT = 8;
static final int CONSUMER_COUNT = 2;
public static void main(String[] args) throws InterruptedException {
IBlockedQueue<Product> queue = new SimpleBlockedQueue<>(controller);
List<Producer> producers = new ArrayList<>();
List<Consumer> consumers = new ArrayList<>();
for (int i = 0; i < PRODUCER_COUNT; i++) {
Producer p = new Producer(i + 1, queue);
p.start();
producers.add(p);
}
for (int i = 0; i < CONSUMER_COUNT; i++) {
Consumer c = new Consumer(i + 1, queue);
c.start();
consumers.add(c);
}
for (Producer p: producers) {
p.join();
}
System.out.println("--All products are produced--");
for (Consumer c: consumers) {
c.join();
}
System.out.println("--All products are consumed--");
System.out.printf("Total produced: %d\n", queue.getTotalProduced());
System.out.printf("Total consumed: %d\n", queue.getTotalConsumed());
}
运行结果
.......
Queue now has 9 items
Consumer 2 consumed item 2093
Queue now has 10 items
Producer 2 produced item 2099
--All products are produced--
Queue now has 9 items
Consumer 1 consumed item 5096
Queue now has 8 items
Consumer 2 consumed item 2094
Queue now has 7 items
Consumer 1 consumed item 5097
Queue now has 6 items
Consumer 2 consumed item 2095
Queue now has 5 items
Consumer 1 consumed item 5098
Queue now has 4 items
Consumer 2 consumed item 2096
Queue now has 3 items
Consumer 1 consumed item 5099
Queue now has 2 items
Consumer 2 consumed item 2097
Queue now has 1 items
Consumer 1 consumed item 2098
Queue now has 0 items
Consumer 2 consumed item 2099
我们看到，虽然生产者已经生产完了，可消费者还在继续等待，程序并没有停下来。此时我们不能使用interupt()等方法去强行停止消费者，因为此时消费者可能还在消费产品（我们并不知道消费者什么时候消费完）。如果我们想让程序在消费者消费完队列中剩下的产品后停下，可以进行如下改动
5. 安全地停止程序
增加一个Controller类，这个类非常简单，实际上就是包装了一个boolean
public class Controller {
private boolean done = false;
public boolean isDone() {
return done;
}
public void shutdown() {
done = true;
}
}
将Controller传入队列构造函数，并在出队时多一个终止条件
public class SimpleBlockedQueue<T> implements IBlockedQueue<T> {
...
Controller c;
public SimpleBlockedQueue(Controller c) {
this.c = c;
}
public synchronized T dequeue(Function<Integer, Void> onSuccess) throws InterruptedException {
while (queue.size() == 0 && ! c.isDone()) {
this.wait(); //wait for producer
}
if (queue.size() == 0 && c.isDone()) {
return null;
}
//dequeue
...
}
...
}
并对main和Consumer做相应改动
public class Test {
static Controller controller = new Controller();
...
public static void main(String[] args) throws InterruptedException {
IBlockedQueue<Product> queue = new SimpleBlockedQueue<>(controller);
List<Producer> producers = new ArrayList<>();
List<Consumer> consumers = new ArrayList<>();
for (int i = 0; i < PRODUCER_COUNT; i++) {
Producer p = new Producer(i + 1, queue);
p.start();
producers.add(p);
}
for (int i = 0; i < CONSUMER_COUNT; i++) {
Consumer c = new Consumer(i + 1, queue);
c.start();
consumers.add(c);
}
for (Producer p: producers) {
p.join();
}
System.out.println("--All products are produced--");
controller.shutdown(); // <-------add this line
for (Consumer c: consumers) {
c.join();
}
System.out.println("--All products are consumed--");
System.out.printf("Total produced: %d\n", queue.getTotalProduced());
System.out.printf("Total consumed: %d\n", queue.getTotalConsumed());
}
...
public static class Consumer extends Thread {
...
@Override
public void run() {
try{
while(true) {
sleep(CONSUME_TIME);
Product p = queue.dequeue(size -> {
System.out.printf("Queue now has %d items\n", size);
return null;
});
if (p == null) { // no more products
break;
}
System.out.printf("Consumer %d consumed item %d\n", id, p.id);
}
} catch (InterruptedException e) {
this.interrupt();
}
}
}
输出结果
...
Producer 6 produced item 6099
Queue now has 9 items
Consumer 2 consumed item 4095
Queue now has 10 items
Producer 4 produced item 4099
--All products are produced--
Queue now has 9 items
Consumer 1 consumed item 6096
Queue now has 8 items
Consumer 2 consumed item 8098
Queue now has 7 items
Consumer 1 consumed item 6097
Queue now has 6 items
Consumer 2 consumed item 4096
Queue now has 5 items
Consumer 1 consumed item 6098
Queue now has 4 items
Consumer 2 consumed item 4097
Queue now has 3 items
Consumer 1 consumed item 4098
Queue now has 2 items
Consumer 2 consumed item 8099
Queue now has 1 items
Consumer 1 consumed item 6099
Queue now has 0 items
Consumer 2 consumed item 4099
--All products are consumed--
Total produced: 800
Total consumed: 800
Process finished with exit code 0
程序在消费者消费完所有产品后顺利结束
6. 使用管程实现
之前的方法有一个不足，那就是notifyAll会把所有的生产者和消费者全都唤醒，而这并不必要。生产者只需唤醒消费者，消费者只需唤醒生产者。那么怎么只唤醒一部分线程呢，我们可以使用条件变量。
public class CondBlockedQueue<T> implements IBlockedQueue<T>{
final int LENGTH = 10;
Queue<T> queue = new ArrayDeque<>();
int count = 0;
int totalProduced = 0;
int totalConsumed = 0;
Controller c;
final Lock lock = new ReentrantLock();
final Condition canProduce = lock.newCondition();
final Condition canConsume = lock.newCondition();
public CondBlockedQueue(Controller c) {
this.c = c;
}
public void enqueue(T item, Function<Integer, Void> onSuccess) throws InterruptedException {
lock.lock();
try { //注意将代码段放到try里面，确保即使发生异常，也不会导致死锁
while (queue.size() == LENGTH && ! c.isDone()) {
canProduce.await(); //wait for consumer
}
queue.add(item);
canConsume.signalAll(); //notify consumers
totalProduced++;
if (onSuccess != null) {
onSuccess.apply(this.queue.size());
}
} finally {
lock.unlock();
}
}
public T dequeue(Function<Integer, Void> onSuccess) throws InterruptedException {
lock.lock();
try {
while (queue.size() == 0 && ! c.isDone()) {
canConsume.await(); //wait for producer
}
if (queue.size() == 0 && c.isDone()) {
return null;
}
T item = queue.remove();
canProduce.signalAll(); //notify producers
totalConsumed++;
if (onSuccess != null) {
onSuccess.apply(this.queue.size());
}
return item;
} finally {
lock.unlock();
}
}
public int getTotalProduced() {
return totalProduced;
}
public int getTotalConsumed() {
return totalConsumed;
}
}
我们用lock替换掉了syncronized关键字，并且使用了两个条件变量canProduce和canConsume，用await和signalAll替换了wait和notifyAll。这样就可以将线程根据他们等待的条件，把他们分成两类，每次只唤醒其中的一类即可。
7. 完整源代码
https://github.com/jsc723/java-producer-consumer
扫一扫
1406
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
