文章目录
RabbitMQ + SpringBoot “可靠性投递” 和 “幂等性” 探讨
本章我们来探讨一下消息的可靠性投递和幂等性, 并以发送邮件为场景模拟.
引言
消息的幂等性
消息永远不会被消费多次. 实现方式通常有:
- [本文采用的方式] 唯一 ID + 指纹码机制, 利用数据库的主键去重. 高并发下可能需要 Hash 分库分表.
SELECT COUNT(1) FROM T_ORDER WHERE ID = 唯一 ID + 指纹码;
- 利用 Redis 的原子性. 哨兵 + 集群模式 (博文链接), 这种方式需要定期将 Redis 的记录同步到数据库中.
消息的可靠性投递
即消息如何保障 100% 投递成功.
首先, 什么是生产端的可靠性投递?
- 保障消息的成功发出;
- 保障 MQ Broker 的成功接收: 借助 ReturnListener;
- 发送端收到 MQ Broker 确认应答 (ACK, NACK): 借助 ConfirmListerner;
消费端需要做什么来保证消息的 100% 消费?
- 完善的消息进行补偿机制: 定时任务抓取没有应答的消息;
解决方案通常也有两种:
-
[本文采用的方式] 消息落库, 对消息状态进行打标;
- 生产者投递消息后, 将消息落库;
- 消费者消费并回执;
- 生产者通过 ConfirmListener 接收 ACK / NACK 对消息状态更新;
- 开启定时任务抓取投递失败消息进行重发;
-
消息的延迟投递, 做二次确认, 回调检查;
-
生产者发送消息后, 稍后再发一条确认的延迟消息;
-
消费者监听到消息并消费后, 会发一条确认消息到 MQ Broker;
-
MQ Broker 收到消费者的确认消息后, 消息落库, 随后与生产者发送的确认的延迟消息内容比对是否收到了这条消息:
- 如果收到就落库
- ;如果没收到就通知生产者重发
-
回调服务 (Callback Service) 接收消费者的 ack
-
整个过程对数据库的操作很少, 并且对数据库的操作交由回调服务, 与核心链路独立.
实现
本文基于发送邮件的场景来介绍如何实现可靠性投递和幂等性, 本质上是模拟一个通信模块, 消息的生产者 (Producer) 申请请求投递邮件, 将请求消息发送到消费端, 消费端 (Consumer) 接收到邮件发送请求, 则发起邮件. 整个链路就是这样.
首先我们需要做到的是:
- 在生产端, 发送消息前对消息记录落库. 需要保证消息 100% 投递到 Broker, 对投递状态打标 (失败);
- 在生产端, 如果路由不到指定的队列, 也需要消费端补偿投递成功但是一段时间内没有消费的消息;
- 在消费端, 如果消息被成功消费, 更新消息投递记录的状态 (成功);
- 在消费端, 如果消息消费失败 (本案例中, 可能是由于邮件发送服务调用失败), 更新消息投递记录 (失败);
- 在消费端, 开启定时任务定时拉取 (失败) 的记录进行重发补偿;
介绍几个核心类的实现, 相关要点都在代码注释中.
完整代码请参考: 代码仓库
server:
port: 17005
servlet:
context-path: /demo-rabbitmq-springboot-idempotence-reliability
spring:
application:
name: demo-rabbitmq-springboot-idempotence-reliability
datasource:
url: jdbc:mysql://118.24.109.247:3306/demo-rabbitmq
password: "!GFLiKe0"
username: root
driver-class-name: com.mysql.cj.jdbc.Driver
mail:
host: smtp.163.com
username: caplike@163.com
password: AIABWEWTQVNQQRVX
default-encoding: UTF-8
properties:
mail:
smtp:
auth: true
ssl:
enable: true
starttls:
enable: true
required: true
# ~ RabbitMQ Configuration
rabbitmq:
host: 118.24.109.247
publisher-confirm-type: correlated
publisher-returns: true
listener:
simple:
acknowledge-mode: manual
prefetch: 10
direct:
acknowledge-mode: manual
# ~ MyBatis Configuration
mybatis:
configuration:
map-underscore-to-camel-case: true
# ~ Logging.Level
logging:
level:
cn.caplike.demo.rabbitmq.springboot.idempotence_reliability: debug
# ~ Custom Configurations
# ---------------------------- ------------------------------------------------------------------------------------------
# ~ Rabbit Broker Declaration
message:
rabbit:
mail:
queue:
name: demo.idempotence.reliability.mail
exchange:
name: demo.idempotence.reliability.mail
type: direct
routing-key: demo.idempotence.reliability.mail
Producer - RabbitConfiguration 配置类
启用生产端投递确认机制和监听路由不到的消息, 需要在配置类配置:
值得注意的是, 当 Exchange 存在, 但是 RoutingKey 错误的 “不可路由” 的消息, ConfirmCallback
的 ack = true, 同时 ReturnCallback
也会触发. 对于这类不可路由消息, 应该由消费端定时监听 投递成功但是一定时间之后还没有被消费的记录, 并进行补偿 (TODO).
/**
* Description: RabbitMQ 配置类
*
* @author LiKe
* @version 1.0.0
* @date 2020-09-01 17:10
*/
@Slf4j
@Configuration
public class RabbitConfiguration {
private CachingConnectionFactory connectionFactory;
private MessageDeliveryLogService messageDeliveryLogService;
/**
* 消息的确认指的是生产者投递消息后, 如果 Broker 收到消息, 给生产者应答.<br>
* 生产者接收应答用来确认消息是否正常的发送到 Broker, 这种方式也是 Producer 端消息的 <strong>可靠性投递</strong> 的核心保障.
*/
private final RabbitTemplate.ConfirmCallback confirmCallback = (correlationData, ack, cause) -> {
final String messageId = Optional.ofNullable(correlationData).orElseThrow(() -> new RuntimeException("correlationData is null!")).getId();
if (ack) {
// ~ [ ! ] 如果消息发送到 Exchange 成功, 但是 routingKey 错误, 则会触发 ReturnCallback,
// ※ 只要 Exchange 正确, 消息成功发送到 Broker, confirmCallback 的 ack = {@code true}
log.info("Producer :: 消息发送到 Broker - 成功.");
// ~ 插入成功发送的日志记录
messageDeliveryLogService.deliverySuccess(messageId);
} else {
// ~ [ ! ] 如果生产端发送消息时, 指定了错误的 Exchange, 则消息并没有发送到 Broker, ack = {@code false}
log.error("Producer :: 消息发送到 Broker - 失败. correlationData: {} cause: {}", correlationData, cause);
// ~ 消息都没发送到 Broker
messageDeliveryLogService.deliveryFailed(messageId);
}
};
/**
* 用于处理一些 <strong>不可路由</strong> 的消息.
*/
private final RabbitTemplate.ReturnCallback returnCallback = (message, replyCode, replyText, exchange, routingKey) -> {
// ~ [ ! ] 如果发送到 Exchange 成功, 但是 RoutingKey 不正确:
// ConfirmCallback ack = {@code true}, 但是 ReturnCallback 也会调用; 需要注意 ReturnCallback 和 ConfirmCallback 的调用顺序.
log.error("Producer :: 消息从 Exchange 路由到 Queue 失败: exchange: {}, routingKey: {}, replyCode: {}, replayText: {}, message: {}", exchange, routingKey, replyCode, replyText, message);
final String messageId = MapUtils.getString(message.getMessageProperties().getHeaders(), PublisherCallbackChannel.RETURNED_MESSAGE_CORRELATION_KEY);
// ~ TODO 消息发送到了 Broker 但是不可路由, Consumer 端也应该监控这种投递成功但是隔了一段时间还没有消费的消息
};
@Bean
public RabbitTemplate rabbitTemplate() {
// ~ RabbitTemplate
// -------------------------------------------------------------------------------------------------------------
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
rabbitTemplate.setConfirmCallback(confirmCallback);
// ~ 在消息没有被路由到合适的队列情况下,Broker 会将消息返回给生产者,
// 为 true 时如果 Exchange 根据类型和消息 Routing Key 无法路由到一个合适的 Queue 存储消息,
// Broker 会调用 Basic.Return 回调给 handleReturn(),再回调给 ReturnCallback,将消息返回给生产者。
// 为 false 时,丢弃该消息
rabbitTemplate.setMandatory(Boolean.TRUE);
rabbitTemplate.setReturnCallback(returnCallback);
return rabbitTemplate;
}
// ~ Autowired
// -----------------------------------------------------------------------------------------------------------------
@Autowired
public void setConnectionFactory(CachingConnectionFactory connectionFactory) {
this.connectionFactory = connectionFactory;
}
@Autowired
public void setMessageDeliveryLogService(MessageDeliveryLogService messageDeliveryLogService) {
this.messageDeliveryLogService = messageDeliveryLogService;
}
}
Producer - MailProducer
MailRequest
为定义的邮件请求对象, 通过 JSON 发送.
/**
* Description: 邮件发送端
*
* @author LiKe
* @version 1.0.0
* @date 2020-09-02 13:21
*/
@Slf4j
@RestController
@RequestMapping("/notification")
public class MailProducer {
private RabbitTemplate rabbitTemplate;
private MessageDeliveryLogService messageDeliveryLogService;
@Value("${message.rabbit.mail.exchange.name}")
private String exchange;
@Value("${message.rabbit.mail.routing-key}")
private String routingKey;
@PostMapping("/mail")
public void send() {
final String messageId = UUID.randomUUID().toString();
final MailRequest mailRequest = MailRequest.builder().messageId(messageId).clientId("local").clientSecret("some-client-secret").receiver("lks.nova@foxmail.com").build();
// ~ 消息落库
// 当消息落库失败? 落库成功再发消息, 如果落库成功消息发送失败, 可追溯.
messageDeliveryLogService.add(
MessageDeliveryLogBuilder.of()
.messageId(messageId).message(mailRequest).exchange(exchange).routingKey(routingKey).status(MessageDeliveryLog.Status.DELIVERING).build()
);
// ~ 关于 correlationId: https://stackoverflow.com/questions/53857051/why-cant-i-get-the-correlationid-on-the-consumer-side-by-using-the-spring-boot
rabbitTemplate.convertAndSend(exchange, routingKey, mailRequest, messagePostProcessor(), new CorrelationData(messageId));
}
// ~ MessagePostProcessor
// -----------------------------------------------------------------------------------------------------------------
private MessagePostProcessor messagePostProcessor() {
return message -> {
final MessageProperties messageProperties = message.getMessageProperties();
messageProperties.setContentType(MessageProperties.CONTENT_TYPE_JSON);
messageProperties.setContentEncoding(StandardCharsets.UTF_8.name());
return message;
};
}
// ~ Autowired
// -----------------------------------------------------------------------------------------------------------------
@Autowired
public void setMessageDeliveryLogService(MessageDeliveryLogService messageDeliveryLogService) {
this.messageDeliveryLogService = messageDeliveryLogService;
}
@Autowired
public void setRabbitTemplate(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
}
}
Consumer - RabbitListenerConfiguration
在消费端, 为了接收 JSON 的消息体并自动转换, 需要做额外的配置:
@Configuration
public class RabbitListenerConfiguration implements RabbitListenerConfigurer {
@Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
@Bean
public MessageHandlerMethodFactory messageHandlerMethodFactory() {
final DefaultMessageHandlerMethodFactory messageHandlerMethodFactory = new DefaultMessageHandlerMethodFactory();
messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter());
return messageHandlerMethodFactory;
}
@Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
}
Consumer - MailRequestMessageListener
@Slf4j
@Component
public class MailRequestMessageListener {
/**
* 投递日志服务接口
*/
private MessageDeliveryLogService messageDeliveryLogService;
/**
* 邮件发送服务接口
*/
private MailSendingService mailSendingService;
@RabbitListener(
bindings = @QueueBinding(
value = @Queue(value = "${message.rabbit.mail.queue.name}", autoDelete = "true", durable = "false"),
exchange = @Exchange(value = "${message.rabbit.mail.exchange.name}", type = "${message.rabbit.mail.exchange.type}", autoDelete = "true", durable = "false"),
key = "${message.rabbit.mail.routing-key}"
)
)
public void consume(
Channel channel,
@Payload MailRequest mailRequest, @Headers Map<String, Object> headers
) throws IOException {
final String messageId = mailRequest.getMessageId();
final MessageDeliveryLog messageDeliveryLog = messageDeliveryLogService.get(messageId);
if (messageDeliveryLog.hasConsumed()) {
log.info("Consumer :: 消息 ({}) 已被消费...", messageId);
return;
}
// ~ Confirm
final Long deliveryTag = MapUtils.getLong(headers, AmqpHeaders.DELIVERY_TAG);
try {
// ~ Send Mail
mailSendingService.send(
MailRequest.builder()
.clientId(mailRequest.getClientId()).clientSecret(mailRequest.getClientSecret()).receiver(mailRequest.getReceiver()).build()
);
} catch (Exception any) {
// ~ Mail sending fail, NACK:
// 1. 告诉 Broker 丢弃这条消息 (default);
// 2. 用死信队列处理 (require: requeue = false);
// 3. 用定时任务补偿 (本例采用)
// 4. 重回队列 (requeue = true);
// Reference: https://blog.youkuaiyun.com/sun_tantan/article/details/88667102
log.error("Consumer :: mail sending failed, cause: {}", any.getMessage());
messageDeliveryLogService.deliveryFailed(messageId);
channel.basicNack(deliveryTag, false, false);
return;
}
// ~ Mail sending success, ACK: 告诉 Broker 这条消息已经被消费
// TODO 当消息状态更新失败? 发送到另外一个 Exchange 进行记录, 补偿
messageDeliveryLogService.consumedSuccess(messageId);
channel.basicAck(deliveryTag, Boolean.FALSE);
}
// ~ Autowired
// -----------------------------------------------------------------------------------------------------------------
@Autowired
public void setMessageDeliveryLogService(MessageDeliveryLogService messageDeliveryLogService) {
this.messageDeliveryLogService = messageDeliveryLogService;
}
@Autowired
public void setMailSendingService(MailSendingService mailSendingService) {
this.mailSendingService = mailSendingService;
}
}
Consumer - 定时补偿
@Slf4j
@Service
public class ScheduledMailResendingService {
private MessageDeliveryLogService messageDeliveryLogService;
private RabbitTemplate rabbitTemplate;
/**
* Description: 每隔一定时间 (30s) 抓取投递失败的消息执行重新投递
*
* @return void
* @author LiKe
* @date 2020-09-03 09:45:17
*/
@Scheduled(cron = "0/30 * * * * ?")
public void execute() {
log.info("Producer :: 开始执行定时任务 - 重新投递");
messageDeliveryLogService.fetchDeliveryFailed().forEach(messageDeliveryLog -> {
final String messageId = messageDeliveryLog.getMessageId();
if (messageDeliveryLog.getRetryCount() <= 3) {
rabbitTemplate.convertAndSend(
messageDeliveryLog.getExchange(),
messageDeliveryLog.getRoutingKey(),
JSON.parseObject(new String(Base64.getDecoder().decode(messageDeliveryLog.getMessage().getBytes())), MailRequest.class),
new CorrelationData(messageId)
);
messageDeliveryLogService.updateResendingStatus(messageId);
log.info("Producer :: 第 {} 次重新投递消息 (messageId: {})", messageDeliveryLog.getRetryCount() + 1, messageId);
}
log.error("Producer :: 消息 (messageId: {}) 超过重试次数", messageId);
});
}
// ~ Autowired
// -----------------------------------------------------------------------------------------------------------------
@Autowired
public void setMessageDeliveryLogService(MessageDeliveryLogService messageDeliveryLogService) {
this.messageDeliveryLogService = messageDeliveryLogService;
}
@Autowired
public void setRabbitTemplate(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
}
}
Consumer - 邮件发送服务
这里采用 thymeleaf 作为邮件模板, 可以参考 代码仓库.
public class MailSendingService {
private static final String MAIL_SENDER = "caplike@163.com";
private JavaMailSender mailSender;
/**
* {@code Thymeleaf} 模板引擎
*/
private TemplateEngine templateEngine;
/**
* Description: 发送邮件
*
* @param mailRequest 邮件内容实体
* @return void
* @throws RuntimeException 如果发送邮件异常
* @author LiKe
* @date 2020-09-01 17:01:10
*/
public void send(MailRequest mailRequest) {
try {
MimeMessage mimeMessage = mailSender.createMimeMessage();
MimeMessageHelper mimeMessageHelper = new MimeMessageHelper(mimeMessage, true);
mimeMessageHelper.setFrom(MAIL_SENDER);
mimeMessageHelper.setTo(mailRequest.getReceiver());
mimeMessageHelper.setSubject("Client Secret Notification!");
mimeMessageHelper.setText(
templateEngine.process(
"mail/template-client-secret-request",
new Context(
Locale.CHINESE,
MapUtils.putAll(new HashMap<>(), new String[]{
"clientId", StringUtils.upperCase(mailRequest.getClientId()),
"clientSecret", UUID.randomUUID().toString()
})
)
), true);
mailSender.send(mimeMessage);
} catch (MessagingException ex) {
throw new RuntimeException(ex);
}
}
// ~ Autowired
// -----------------------------------------------------------------------------------------------------------------
@Autowired
public void setMailSender(JavaMailSender mailSender) {
this.mailSender = mailSender;
}
@Autowired
public void setTemplateEngine(TemplateEngine templateEngine) {
this.templateEngine = templateEngine;
}
}
后记
核心代码就是这么多了. 其他的 MyBatis 相关代码请参考 代码仓库 吧.
ReturnCallback
接收的不可路由的消息, 实际生产环境中不太可能遇得到.
TODO. 下一篇介绍: 通过死信队列处理 nack 的消息.
Refernece
- RabbitMQ 日常爬坑分享
- RabbitMQ (三) 手动 Ack 确认
- SpringBoot 定时任务 Scheduled
- SpringBoot 设置手动确认 ACK: Channel shutdown 异常
- RabbitMQ 怎么确认消费者已经消费了消息
- https://www.jianshu.com/p/dca01aad6bc8