互联网三高-高性能之Disruptor基础原理

最新推荐文章于 2025-07-09 21:53:25 发布

置顶红云梦

最新推荐文章于 2025-07-09 21:53:25 发布

阅读量861

点赞数 28

CC 4.0 BY-SA版权

分类专栏：工作总结学习总结文章标签： java 并发编程

本文链接：https://blog.youkuaiyun.com/hong4760/article/details/9277247

学习总结同时被 2 个专栏收录

12 篇文章

订阅专栏

工作总结

11 篇文章

订阅专栏

1 基本介绍

Disruptor 是由英国 LMAX 交易所开发的高性能无锁并发框架，专为多线程场景设计，旨在解决传统队列（如 ArrayBlockingQueue）的锁竞争、内存分配等性能瓶颈问题‌。其单线程处理能力可达 ‌每秒600万事件‌，广泛应用于金融交易、日志处理等高吞吐场景‌。

2 快速入门

1）导入依赖

<dependency>
	<groupId>com.lmax</groupId>
	<artifactId>disruptor</artifactId>
	<version>3.4.4</version>
</dependency>

2）创建Event工厂类，用来创建Event实例对象

// Event对象，普通实体类
@Data
public class OrderEvent {

    /**
     * 订单ID
     */
    private Long oderId;
}

/**
 * @description: 创建Event工厂类
 * @author: panhong
 * @date: 2025/4/10
 */
public class OrderEventFactory implements EventFactory<OrderEvent> {
    @Override
    public OrderEvent newInstance() {
        return new OrderEvent();
    }
}

3）创建事件监听类，用来处理数据

/**
 * @description: 事件监听类，用来处理数据
 * @author: panhong
 * @date: 2025/4/10
 */
public class OrderEventHandler implements EventHandler<OrderEvent> {

    @Override
    public void onEvent(OrderEvent event, long sequence, boolean endOfBatch) throws Exception {
        System.out.println("消费数据：" + event.getOderId());
    }
}

4）创建Disruptor核心组件

① 实例化Disruptor对象，核心参数：Event工厂类，ringBuffer大小，线程池，发布者类型（单实例，多实例），等待策略

② 添加消费者监听

③ 启动Disruptor对象

④ 创建生产者组件

⑤ 使用完成，需要关闭资源

public class Main {

    public static void main(String[] args) {
        OrderEventFactory eventFactory = new OrderEventFactory();
        int ringBufferSize = 1024 * 1024;
        ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

        // 创建Disruptor对象
        Disruptor<OrderEvent> disruptor = new Disruptor<>(eventFactory, ringBufferSize,
                executorService, ProducerType.SINGLE,
                new BlockingWaitStrategy());

        // 添加消费者监听
        disruptor.handleEventsWith(new OrderEventHandler());

        // 启动 disruptor
        disruptor.start();

        // 获取实际存储数据的容器：RingBuffer
        RingBuffer<OrderEvent> ringBuffer = disruptor.getRingBuffer();

        // 创建生产者组件
        OrderEventProducer producer = new OrderEventProducer(ringBuffer);

        ByteBuffer bb = ByteBuffer.allocate(8);

        for(int i=0; i<100; i++) {
            bb.putLong(0, i);

            producer.sendData(bb);
        }

        // 关闭操作
        disruptor.shutdown();
        executorService.shutdown();
    }
}

5）编写生产者组件，向Disruptor容器中投递数据

使用ringBuffer完成提交发布

public class OrderEventProducer {

    private RingBuffer<OrderEvent> ringBuffer;

    public OrderEventProducer(RingBuffer<OrderEvent> ringBuffer) {
        this.ringBuffer = ringBuffer;
    }

    public void sendData(ByteBuffer data) {
        // 从RingBuffer中获取一个可用序号
        long sequence = ringBuffer.next();
        try {
            // 根据这个序号获取对象，此时的对象是一个没有赋值的空对象
            OrderEvent orderEvent = ringBuffer.get(sequence);
            // 赋值
            orderEvent.setOderId(data.getLong(0));
        } finally {
            // 提交发布操作
            ringBuffer.publish(sequence);
        }
    }
}

3 高级应用

3.1 并行计算

① 串行操作使用链式调用

disruptor.handleEventsWith(handler).handleEventsWith(handler)；

disruptor.handleEventsWith(new Handler1()).handleEventsWith(new Handler2()).handleEventsWith(new Handler3());

② 并行操作使用单独调用

disruptor.handleEventsWith(handler);
disruptor.handleEventsWith(handler);

或者

disruptor.handleEventsWith(handler1,handler2);

disruptor.handleEventsWith(new Handler1(),new Handler2(),new Handler3());

③ 菱形操作（串、并行操作）

disruptor.handleEventsWith(new Handler1()).handleEventsWith(new Handler2(),new Handler3())
               .handleEventsWith(new Handler4());

④ 六边形操作

注意点：有几个Handler就要几个线程（disruptor中的线程池个数设置）

Handler1 h1 = new Handler1();
Handler2 h2 = new Handler2();
Handler3 h3 = new Handler3();
Handler4 h4 = new Handler4();
Handler5 h5 = new Handler5();
disruptor.handleEventsWith(h1, h4);
disruptor.after(h1).handleEventsWith(h2);
disruptor.after(h4).handleEventsWith(h5);
disruptor.after(h2, h5).handleEventsWith(h3);

3.2 多生产者多消费者模型

1）构建多消费者工作池 WorkerPool

//1 创建RingBuffer
RingBuffer<Order> ringBuffer =
		RingBuffer.create(ProducerType.MULTI,
				new EventFactory<Order>() {
					public Order newInstance() {
						return new Order();
					}
				},
				1024*1024,
				new YieldingWaitStrategy());

//2 通过ringBuffer 创建一个屏障
SequenceBarrier sequenceBarrier = ringBuffer.newBarrier();

//3 创建多个消费者数组:
Consumer[] consumers = new Consumer[10];
for(int i = 0; i < consumers.length; i++) {
	consumers[i] = new Consumer("C" + i);
}

//4 构建多消费者工作池
WorkerPool<Order> workerPool = new WorkerPool<Order>(
		ringBuffer,
		sequenceBarrier,
		new EventExceptionHandler(),
		consumers);

//5 设置多个消费者的sequence序号 用于单独统计消费进度, 并且设置到ringbuffer中
ringBuffer.addGatingSequences(workerPool.getWorkerSequences());

//6 启动workerPool
workerPool
.start(Executors.newFixedThreadPool(5));

final CountDownLatch latch = new CountDownLatch(1);

for(int i = 0; i < 100; i++) {
	final Producer producer = new Producer(ringBuffer);
	new Thread(new Runnable() {
		public void run() {
			try {
				latch.await();
			} catch (Exception e) {
				e.printStackTrace();
			}
			for(int j = 0; j<100; j++) {
				producer.sendData(UUID.randomUUID().toString());
			}
		}
	}).start();
}

2）创建多个生产者

for(int i = 0; i < 100; i++) {
	final Producer producer = new Producer(ringBuffer);
	new Thread(new Runnable() {
		public void run() {
			try {
				latch.await();
			} catch (Exception e) {
				e.printStackTrace();
			}
			for(int j = 0; j<100; j++) {
				producer.sendData(UUID.randomUUID().toString());
			}
		}
	}).start();
}

4 核心原理

4.1 无锁环形队列（RingBuffer）

Disruptor通过环形数组结构（RingBuffer）实现高效数据存储和循环覆盖机制，预分配内存空间以避免频繁GC开销‌。生产者通过‌Sequencer‌（单生产者：SingleProducerSequencer；多生产者：MultiProducerSequencer）向RingBuffer申请序列号并发布事件，消费者根据序列号顺序处理数据。