RocketMQ 常见功能详解与 Spring Cloud 集成 实现

于 2025-07-05 09:48:45 发布
阅读量394 收藏 9
点赞数 4
CC 4.0 BY-SA版权
分类专栏： MQ 文章标签： java-rocketmq rocketmq spring cloud
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/csdn_tom_168/article/details/149004479

RocketMQ 常见功能详解与 Spring Cloud 集成 实现

一、Spring Cloud Stream + RocketMQ 集成

1. Maven 依赖

<dependencies>
    <!-- Spring Cloud Stream RocketMQ Binder -->
    <dependency>
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-stream-rocketmq</artifactId>
        <version>2022.0.0.0-RC2</version>
    </dependency>
    
    <!-- Spring Boot Web -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    
    <!-- Lombok -->
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <optional>true</optional>
    </dependency>
</dependencies>

2. 配置文件 (application.yml)

spring:
  application:
    name: rocketmq-demo
  cloud:
    stream:
      rocketmq:
        binder:
          name-server: localhost:9876 # RocketMQ NameServer地址
          group: spring-cloud-group
      bindings:
        # 普通消息通道
        normal-output:
          destination: NORMAL_TOPIC
          content-type: application/json
          producer:
            group: normal-producer-group
        normal-input:
          destination: NORMAL_TOPIC
          content-type: application/json
          group: normal-consumer-group
          consumer:
            tags: "tagA"
        
        # 顺序消息通道
        order-output:
          destination: ORDER_TOPIC
          content-type: application/json
          producer:
            group: order-producer-group
            messageQueueSelector: orderSelector # 自定义队列选择器
        order-input:
          destination: ORDER_TOPIC
          content-type: application/json
          group: order-consumer-group
          consumer:
            messageModel: orderly # 顺序消费模式
        
        # 事务消息通道
        transaction-output:
          destination: TX_TOPIC
          content-type: application/json
          producer:
            group: tx-producer-group
            transactionListener: myTransactionListener # 事务监听器
        
        # 延迟消息通道
        delay-output:
          destination: DELAY_TOPIC
          content-type: application/json
          producer:
            group: delay-producer-group
        delay-input:
          destination: DELAY_TOPIC
          content-type: application/json
          group: delay-consumer-group
        
        # 广播消息通道
        broadcast-input:
          destination: BROADCAST_TOPIC
          content-type: application/json
          group: broadcast-consumer-group
          consumer:
            messageModel: broadcasting # 广播模式

二、核心功能实现

1. 普通消息

生产者
@RestController
@RequiredArgsConstructor
public class NormalProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendNormal")
    public String sendNormal() {
        Map<String, Object> message = new HashMap<>();
        message.put("id", UUID.randomUUID().toString());
        message.put("content", "普通消息内容");
        message.put("timestamp", System.currentTimeMillis());
        
        // 发送消息
        boolean result = streamBridge.send("normal-output", message);
        return result ? "普通消息发送成功" : "发送失败";
    }
}
消费者
@Component
@Slf4j
public class NormalConsumer {

    @Bean
    public Consumer<Message<Map<String, Object>>> normal-input() {
        return message -> {
            Map<String, Object> payload = message.getPayload();
            log.info("收到普通消息: ID={}, 内容={}, 时间={}", 
                payload.get("id"), 
                payload.get("content"),
                new Date((Long) payload.get("timestamp"))
            );
        };
    }
}

2. 顺序消息

自定义队列选择器
@Component("orderSelector")
public class OrderMessageQueueSelector implements MessageQueueSelector {

    @Override
    public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
        // arg 为订单ID，确保同一订单的消息发送到同一个队列
        String orderId = (String) arg;
        int index = Math.abs(orderId.hashCode()) % mqs.size();
        return mqs.get(index);
    }
}
生产者
@RestController
@RequiredArgsConstructor
public class OrderProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendOrder/{orderId}")
    public String sendOrder(@PathVariable String orderId) {
        for (int i = 1; i <= 5; i++) {
            Map<String, Object> message = new HashMap<>();
            message.put("orderId", orderId);
            message.put("step", i);
            message.put("content", "订单步骤-" + i);
            
            // 发送顺序消息，指定shardingKey（orderId）
            streamBridge.send("order-output", 
                MessageBuilder.withPayload(message)
                    .setHeader(RocketMQHeaders.SHARDING_KEY, orderId)
                    .build()
            );
        }
        return "订单消息发送成功: " + orderId;
    }
}
消费者
@Component
@Slf4j
public class OrderConsumer {

    @Bean
    public Consumer<Message<Map<String, Object>>> order-input() {
        return message -> {
            Map<String, Object> payload = message.getPayload();
            log.info("顺序消费: 订单ID={}, 步骤={}, 内容={}", 
                payload.get("orderId"),
                payload.get("step"),
                payload.get("content")
            );
        };
    }
}

3. 事务消息

事务监听器
@Component("myTransactionListener")
public class MyTransactionListener implements RocketMQLocalTransactionListener {

    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message msg, Object arg) {
        try {
            // 执行本地事务
            log.info("执行本地事务...");
            // 模拟业务处理
            Thread.sleep(500);
            
            // 随机返回状态模拟不同结果
            return new Random().nextBoolean() ? 
                RocketMQLocalTransactionState.COMMIT : 
                RocketMQLocalTransactionState.ROLLBACK;
        } catch (Exception e) {
            return RocketMQLocalTransactionState.UNKNOWN;
        }
    }

    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message msg) {
        log.info("执行事务回查...");
        // 这里应该查询数据库判断本地事务状态
        return RocketMQLocalTransactionState.COMMIT;
    }
}
生产者
@RestController
@RequiredArgsConstructor
public class TransactionProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendTransaction")
    public String sendTransaction() {
        Map<String, Object> message = new HashMap<>();
        message.put("txId", UUID.randomUUID().toString());
        message.put("amount", new Random().nextInt(1000));
        
        // 发送事务消息
        streamBridge.send("transaction-output", 
            MessageBuilder.withPayload(message)
                .setHeader(RocketMQHeaders.TRANSACTION_ID, "TX_" + System.currentTimeMillis())
                .build()
        );
        
        return "事务消息已发送";
    }
}

4. 延迟消息

生产者
@RestController
@RequiredArgsConstructor
public class DelayProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendDelay/{level}")
    public String sendDelay(@PathVariable int level) {
        Map<String, Object> message = new HashMap<>();
        message.put("id", UUID.randomUUID().toString());
        message.put("delayLevel", level);
        message.put("sendTime", System.currentTimeMillis());
        
        // 发送延迟消息
        streamBridge.send("delay-output", 
            MessageBuilder.withPayload(message)
                .setHeader(RocketMQHeaders.DELAY, level) // 设置延迟级别
                .build()
        );
        
        return "延迟消息发送成功，级别: " + level;
    }
}
消费者
@Component
@Slf4j
public class DelayConsumer {

    @Bean
    public Consumer<Message<Map<String, Object>>> delay-input() {
        return message -> {
            Map<String, Object> payload = message.getPayload();
            long sendTime = (Long) payload.get("sendTime");
            long now = System.currentTimeMillis();
            long delay = now - sendTime;
            
            log.info("收到延迟消息: ID={}, 延迟级别={}, 实际延迟={}ms", 
                payload.get("id"),
                payload.get("delayLevel"),
                delay
            );
        };
    }
}

5. 广播消息

生产者
@RestController
@RequiredArgsConstructor
public class BroadcastProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendBroadcast")
    public String sendBroadcast() {
        Map<String, Object> message = new HashMap<>();
        message.put("id", UUID.randomUUID().toString());
        message.put("content", "广播消息内容");
        
        streamBridge.send("broadcast-output", message);
        return "广播消息发送成功";
    }
}
消费者
@Component
@Slf4j
public class BroadcastConsumer {

    @Value("${server.port}")
    private String port;

    @Bean
    public Consumer<Map<String, Object>> broadcast-input() {
        return message -> {
            log.info("实例[{}] 收到广播消息: ID={}, 内容={}", 
                port,
                message.get("id"),
                message.get("content")
            );
        };
    }
}

6. 消息过滤（SQL92）

生产者
@RestController
@RequiredArgsConstructor
public class FilterProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendFilter")
    public String sendFilter() {
        // 创建VIP用户消息
        Map<String, Object> vipMessage = new HashMap<>();
        vipMessage.put("id", "VIP-" + UUID.randomUUID());
        vipMessage.put("userType", "vip");
        vipMessage.put("age", 30);
        vipMessage.put("content", "VIP专属消息");
        
        // 创建普通用户消息
        Map<String, Object> normalMessage = new HashMap<>();
        normalMessage.put("id", "NORMAL-" + UUID.randomUUID());
        normalMessage.put("userType", "normal");
        normalMessage.put("age", 20);
        normalMessage.put("content", "普通用户消息");
        
        // 发送消息
        streamBridge.send("filter-output", vipMessage);
        streamBridge.send("filter-output", normalMessage);
        
        return "过滤消息发送成功";
    }
}
消费者配置
spring:
  cloud:
    stream:
      bindings:
        filter-input:
          destination: FILTER_TOPIC
          content-type: application/json
          group: filter-consumer-group
          consumer:
            sql: "userType = 'vip' AND age > 25" # SQL92过滤条件
消费者
@Component
@Slf4j
public class FilterConsumer {

    @Bean
    public Consumer<Map<String, Object>> filter-input() {
        return message -> {
            log.info("收到过滤消息: ID={}, 用户类型={}, 年龄={}, 内容={}", 
                message.get("id"),
                message.get("userType"),
                message.get("age"),
                message.get("content")
            );
        };
    }
}

7. 批量消息

生产者
@RestController
@RequiredArgsConstructor
public class BatchProducerController {

    private final StreamBridge streamBridge;

    @GetMapping("/sendBatch/{count}")
    public String sendBatch(@PathVariable int count) {
        List<Map<String, Object>> messages = new ArrayList<>();
        for (int i = 0; i < count; i++) {
            Map<String, Object> message = new HashMap<>();
            message.put("id", UUID.randomUUID().toString());
            message.put("index", i);
            message.put("content", "批量消息-" + i);
            messages.add(message);
        }
        
        // 发送批量消息
        streamBridge.send("batch-output", messages);
        return "批量消息发送成功，数量: " + count;
    }
}
消费者
@Component
@Slf4j
public class BatchConsumer {

    @Bean
    @SuppressWarnings("unchecked")
    public Consumer<Message<List<Map<String, Object>>>> batch-input() {
        return message -> {
            List<Map<String, Object>> payload = message.getPayload();
            log.info("收到批量消息，数量: {}", payload.size());
            payload.forEach(msg -> 
                log.info(" - 消息ID={}, 索引={}", msg.get("id"), msg.get("index"))
            );
        };
    }
}

三、高级功能实现

1. 消息幂等性处理

@Component
@Slf4j
public class IdempotentConsumer {

    private final Set<String> processedMessages = ConcurrentHashMap.newKeySet();

    @Bean
    public Consumer<Message<Map<String, Object>>> idempotent-input() {
        return message -> {
            String msgId = (String) message.getHeaders().get(RocketMQHeaders.PREFIX + RocketMQHeaders.KEYS);
            
            // 检查消息是否已处理
            if (processedMessages.contains(msgId)) {
                log.warn("消息重复，已跳过: {}", msgId);
                return;
            }
            
            // 处理消息
            Map<String, Object> payload = message.getPayload();
            log.info("处理新消息: ID={}, 内容={}", msgId, payload.get("content"));
            
            // 记录已处理消息
            processedMessages.add(msgId);
        };
    }
}

2. 消费重试与死信队列

spring:
  cloud:
    stream:
      bindings:
        retry-input:
          destination: RETRY_TOPIC
          group: retry-consumer-group
          consumer:
            maxAttempts: 3 # 最大重试次数
            backOffInitialInterval: 1000 # 初始重试间隔(ms)
            backOffMaxInterval: 10000 # 最大重试间隔(ms)
            backOffMultiplier: 2.0 # 重试间隔乘数
            # 死信队列配置
            dlqName: DLQ_RETRY_TOPIC

3. 消息轨迹追踪

@Configuration
public class TracingConfig {

    @Bean
    public Function<Message<?>, Message<?>> tracingFunction() {
        return message -> {
            // 添加追踪信息
            String traceId = "TRACE-" + System.currentTimeMillis();
            return MessageBuilder.fromMessage(message)
                    .setHeader("X-Trace-Id", traceId)
                    .build();
        };
    }
}

四、Spring Cloud Stream 核心概念

1. 绑定器 (Binder)

@Bean
public Binder<MessageChannel, ?, ?> rocketBinder(RocketMQResourceManager resourceManager) {
    return new RocketMQMessageChannelBinder(resourceManager);
}

2. 消息通道 (Message Channels)

public interface CustomChannels {
    
    @Output("custom-output")
    MessageChannel customOutput();
    
    @Input("custom-input")
    SubscribableChannel customInput();
}

3. 消息转换器 (Message Converters)

@Bean
public MessageConverter customMessageConverter() {
    return new CompositeMessageConverter(
        Arrays.asList(
            new MappingJackson2MessageConverter(),
            new ByteArrayMessageConverter()
        )
    );
}

五、最佳实践

1. 生产者配置优化

spring:
  cloud:
    stream:
      rocketmq:
        producer:
          # 发送超时时间(ms)
          sendMessageTimeout: 3000
          # 压缩消息阈值(bytes)
          compressMessageBodyThreshold: 4096
          # 重试次数
          retryTimesWhenSendFailed: 2
          # 异步发送线程池
          asyncSenderExecutor: asyncThreadPool

2. 消费者配置优化

spring:
  cloud:
    stream:
      rocketmq:
        consumer:
          # 每次拉取消息数量
          pullBatchSize: 32
          # 消费线程池最小线程数
          consumeThreadMin: 5
          # 消费线程池最大线程数
          consumeThreadMax: 20
          # 消息消费超时时间(分钟)
          consumeTimeout: 15

3. 异常处理

@Bean
public Consumer<Message<?>> errorConsumer() {
    return message -> {
        try {
            // 业务处理
        } catch (Exception e) {
            // 1. 记录错误日志
            log.error("消息处理失败", e);
            
            // 2. 发送到死信队列
            // 3. 通知监控系统
            // 4. 人工介入处理
        }
    };
}

4. 消息监控

@Bean
public ApplicationRunner monitorRunner(RocketMQTemplate rocketMQTemplate) {
    return args -> {
        // 监控消息堆积
        DefaultMQAdminExt adminExt = new DefaultMQAdminExt();
        adminExt.setNamesrvAddr("localhost:9876");
        adminExt.start();
        
        ClusterInfo clusterInfo = adminExt.examineBrokerClusterInfo();
        // 分析集群状态和消息堆积情况
        
        adminExt.shutdown();
    };
}

六、常见问题解决方案

1. 消息发送失败处理

@Slf4j
@Component
public class SendFailureHandler {

    @Autowired
    private RocketMQTemplate rocketMQTemplate;
    
    public void handleFailure(Message<?> message, Throwable throwable) {
        log.error("消息发送失败", throwable);
        
        // 1. 记录失败消息到数据库
        // 2. 启动定时任务重试
        // 3. 报警通知
        
        // 示例：10秒后重试
        rocketMQTemplate.asyncSendDelaySeconds(
            "RETRY_TOPIC", 
            message, 
            10, 
            new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {
                    log.info("重试发送成功");
                }
                
                @Override
                public void onException(Throwable e) {
                    log.error("重试发送失败", e);
                }
            }
        );
    }
}

2. 消息堆积处理方案

@Bean
public ApplicationRunner messageBacklogMonitor() {
    return args -> {
        ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
        scheduler.scheduleAtFixedRate(() -> {
            // 1. 检查消息堆积量
            long backlog = calculateMessageBacklog();
            
            // 2. 根据堆积量动态调整
            if (backlog > 10000) {
                // 增加消费者实例
                scaleOutConsumers();
                
                // 提高消费线程数
                adjustConsumeThreads(30);
            } else if (backlog < 1000) {
                // 减少消费线程数
                adjustConsumeThreads(10);
            }
        }, 0, 1, TimeUnit.MINUTES);
    };
}

3. 顺序消息消费失败处理

@Component
@Slf4j
public class OrderlyFailureHandler {

    public void handleFailure(Message<?> message, Exception ex) {
        log.error("顺序消息消费失败", ex);
        
        // 1. 记录失败消息
        // 2. 暂停当前队列消费
        // 3. 告警通知人工介入
        
        // 注意：不能跳过消息，否则会破坏顺序性
        // 需要人工处理失败消息后才能恢复消费
    }
}

七、Spring Cloud 集成优势

  1. 声明式编程：通过注解简化消息收发逻辑
  2. 统一编程模型：支持多种消息中间件统一API
  3. 自动化配置：简化RocketMQ客户端配置
  4. 与Spring生态无缝集成：Spring Boot, Spring Security等
  5. 简化事务管理：与Spring事务集成
  6. 弹性扩展：结合Spring Cloud服务发现实现动态扩缩容

通过Spring Cloud Stream集成RocketMQ，开发者可以更专注于业务逻辑实现，同时利用Spring Cloud的丰富生态构建健壮的分布式消息系统。

评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值