RabbitMQ之消费端限流、重回队列、死信队列

最新推荐文章于 2024-05-30 20:00:00 发布

原创最新推荐文章于 2024-05-30 20:00:00 发布 · 1.3k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#java-rabbitmq #rabbitmq #分布式

RabbitMQ实现消息发送的基础组件封装专栏收录该内容

15 篇文章

订阅专栏

本文介绍了RabbitMQ中处理大量消息的策略，包括消费端的QoS限流以避免消息瞬时推送导致客户端崩溃，以及手动ACK和NACK机制。消费端的重回队列用于重试未成功处理的消息，但可能导致循环。死信队列(DLX)则用于处理变为死信的消息，如TTL过期或被拒绝。示例代码展示了如何设置和使用这些功能。

消费端限流、重回队列是在消费端进行设置，死信队列是在队列声明时进行设置

一、消费端限流与重回队列

假设一个场景，rabbitMQ服务器有上万条处理的消息，随便打开一个消费者客户端，巨量的消息瞬间推送过来，可能导致客户端崩溃。

RabbitMQ提供一种 qos 服务质量保证功能，即在非自动确认消息的前提下，如果一定数目的消息未被ACK前，不进行消费新的消息。

java代码通过 channel.basicQos(int prefetchSize, int prefetchCount, boolean global)设置qos，prefetchSize 和 global 两项，rabbitmq 没有实现，暂且不研究；prefetch_count 只有在消费端没有设置为自动 ACK 时才生效。

关于消费端的ACK：

在实际业务中一般不会选择自动ack
消费端的手工ack分为两种ACK和NACK
消费端进行消费的时候，如果由于业务异常我们可以进行日志的记录，然后进行补偿。这种建议回复NACK，不要重回队列

QOS示例代码：

    // 生产端：发送消息
    public static void main(String[] args) throws IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String queueName = "qos_queue";

        String msg = "qos message~~";
        for(int i=0; i<5; i++){
            Map<String, Object> headers = new HashMap<>();
            headers.put("flag", i);
            AMQP.BasicProperties rops = new AMQP.BasicProperties().builder()
                    .contentEncoding("UTF-8")
                    .deliveryMode(2)
                    .headers(headers)
                    .build();
            channel.basicPublish("", queueName, rops, (msg + i).getBytes());
        }

    }

    // 消费端：消费消息
    public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setAutomaticRecoveryEnabled(true);
        factory.setNetworkRecoveryInterval(3000);

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String queueName = "qos_queue";

        channel.queueDeclare(queueName, false, false, false, null);

        QueueingConsumer consumer = new QueueingConsumer(channel);
        // 消费端限流
        channel.basicQos(0,2,false);

        channel.basicConsume(queueName, false, consumer);

        while (true) {
            QueueingConsumer.Delivery delivery = consumer.nextDelivery();
            System.out.println("receive a message:"+new String(delivery.getBody()));
            Thread.sleep(1000);
//            channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);

        }

关于消费端的重回队列

是为了对没有处理成功的消息，把消息重新会投递给broker。
重回队列，会回到队列的尾部
如果一条消息有问题，可能造成该消息一直重复投递，死循环了
在实际应用中，都会关闭重回队列，也就是设置为false

重回队列示例代码：

    // 生产端：发送消息
    public static void main(String[] args) throws IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String exchangeName = "requeue_exchange";
        String routingKey = "requeue.c";
        
        String msg = "requeue message~~";
        for(int i=0; i<5; i++){
            Map<String, Object> headers = new HashMap<>();
            headers.put("flag", i);
            AMQP.BasicProperties rops = new AMQP.BasicProperties().builder()
                    .contentEncoding("UTF-8")
                    .deliveryMode(2)
                    .headers(headers)
                    .build();
            channel.basicPublish(exchangeName, routingKey, rops, (msg + i).getBytes());
        }
    }

    // 消费端：消费消息
    public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setAutomaticRecoveryEnabled(true);
        factory.setNetworkRecoveryInterval(3000);

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String exchangeName = "requeue_exchange";
        String queueName = "requeue_queue";
        String routingKey = "requeue.*";
        String exchangeType = "topic";
        channel.exchangeDeclare(exchangeName, exchangeType, false, false, null);
        channel.queueDeclare(queueName, false, false, false, null);
        channel.queueBind(queueName, exchangeName, routingKey);

        QueueingConsumer consumer = new QueueingConsumer(channel);
        channel.basicConsume(queueName, false, consumer);

        while (true) {
            QueueingConsumer.Delivery delivery = consumer.nextDelivery();
            System.out.println("receive a message:"+new String(delivery.getBody()));
            Thread.sleep(1000);
            if(delivery.getProperties().getHeaders().get("flag").equals(0)){
                // 第三个参数设置为 true 就是重回队列
                channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, true);
            }else {
                channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
            }
        }
    }

二、TTL队列和死信队列

TTL Time to Live的缩写，也就是生存时间

RabbitMQ支持消息的过期时间，在消息发送时可以进行指定。
RabbitMQ支持队列的过期时间，从消息入队列开始计算，只要超过了队列的超时时间配置，那么消息就会自动的清除。

死信队列：DLX, Dead-Letter-Exchange

利用DLX,当消息在队列中变成死信（dead message）之后，它能被重新publish到另一个Exchange,这个Exchange就是DLX。

消息变成死信有以下几种情况

消息被拒绝（basic.reject/basic.nack）并且requeue=false
消息TTL过期
队列达到最大长度，新消息装不下了

死信队列

DLX也是一个正常的Exchange,和一般的Exchange没有区别，它能在任何的队列上被指定，实际上就是设置某个队列的属性。
当这个队列中有死信时，RabbitMQ就会自动将这个消息重新发布到设置的Exchange上去，进而被路由到另一个队列。
可以监听这个队列中消息做相应的处理，这个特性可以弥补RabbitMQ3.0以前支持的immediate参数的功能。

实现：

在正常队列上添加参数：arguments.put("x-dead-letter-exchange","dlx.exchange")，就可以将该队列绑定了一个死信队列。

示例代码：

    
    // 生产端：发送消息
    public static void main(String[] args) throws IOException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String exchangeName = "normal_exchange";
        String routingKey = "normal.c";


        String msg = "dlx message~~";
        for(int i=0; i<5; i++){
            Map<String, Object> headers = new HashMap<>();
            AMQP.BasicProperties rops = new AMQP.BasicProperties().builder()
                    .contentEncoding("UTF-8")
                    .deliveryMode(2)
                    .headers(headers)
                    .build();
            channel.basicPublish(exchangeName, routingKey, rops, (msg + i).getBytes());
        }

    }

    // 消费端：消费消息
    public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("121.43.153.00");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setAutomaticRecoveryEnabled(true);
        factory.setNetworkRecoveryInterval(3000);

        Connection connection = factory.newConnection();
        Channel channel = connection.createChannel();

        String dlxExchangeName = "dlx_exchange";
        String exchangeName = "normal_exchange";
        String queueName = "normal_queue";
        String routingKey = "normal.*";
        String exchangeType = "topic";
        channel.exchangeDeclare(exchangeName, exchangeType, false, false, null);
        Map<String, Object> arguments = new HashMap<>();

        arguments.put("x-dead-letter-exchange", dlxExchangeName);

        channel.queueDeclare(queueName, false, false, false, arguments);
        channel.queueBind(queueName, exchangeName, routingKey);


        String dlxQueueName = "dlx_queue";
        String dlxRoutingKey = "#";
        String dlxExchangeType = "topic";
        channel.exchangeDeclare(dlxExchangeName, dlxExchangeType, false, false, null);
        channel.queueDeclare(dlxQueueName, false, false, false, null);
        channel.queueBind(dlxQueueName, dlxExchangeName, dlxRoutingKey);

        QueueingConsumer consumer = new QueueingConsumer(channel);
        channel.basicConsume(queueName, true, consumer);

        while (true) {
            QueueingConsumer.Delivery delivery = consumer.nextDelivery();
            System.out.println("receive a message:"+new String(delivery.getBody()));
            Thread.sleep(1000);
            channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, false);

        }

    }