SpringbootKaka批量消费报Listener method could not be invoked with the incoming message

最新推荐文章于 2023-07-04 17:00:30 发布
最新推荐文章于 2023-07-04 17:00:30 发布
阅读量9.3k 收藏 9
点赞数 6
分类专栏： kafka 文章标签： kafka java
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/SDAUguo/article/details/121691578
版权
本文详细记录了一起Spring Kafka消费者在批量监听时遇到的类型转换异常问题，通过分析错误日志，发现是由于消息转换错误导致。在尝试多种解决方案后，最终通过设置`spring.kafka.listener.type=batch`修复了问题。此外，还分享了生产者代码、完整的配置信息以及问题出现的背景，为读者提供了解决类似问题的参考。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

结论：在配置文件中加上spring.kafka.listener.type=batch 。

完整报错信息如下：

org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Endpoint handler details:
Method [public void com.kafka.gateway.consumer.heartBeatConsumer.listensGroupPro(org.apache.kafka.clients.consumer.ConsumerRecords<java.lang.String, java.lang.String>,org.springframework.kafka.support.Acknowledgment)]
Bean [com.kafka.gateway.consumer.heartBeatConsumer@3cceaa40]; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot handle message; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [org.apache.kafka.clients.consumer.ConsumerRecords] for GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}], failedMessage=GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}]; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot handle message; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [org.apache.kafka.clients.consumer.ConsumerRecords] for GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}], failedMessage=GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.decorateException(KafkaMessageListenerContainer.java:2636) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2606) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:2567) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:2481) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:2403) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:2282) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:1956) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeIfHaveRecords(KafkaMessageListenerContainer.java:1351) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1342) [spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1234) [spring-kafka-2.8.0.jar:2.8.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_211]
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) ~[na:1.8.0_211]
	at java.util.concurrent.FutureTask.run(FutureTask.java) ~[na:1.8.0_211]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_211]
	Suppressed: org.springframework.kafka.listener.ListenerExecutionFailedException: Restored Stack Trace
		at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.checkAckArg(MessagingMessageListenerAdapter.java:374) ~[spring-kafka-2.8.0.jar:2.8.0]
		at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:352) ~[spring-kafka-2.8.0.jar:2.8.0]
		at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:92) ~[spring-kafka-2.8.0.jar:2.8.0]
		at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:53) ~[spring-kafka-2.8.0.jar:2.8.0]
		at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2586) [spring-kafka-2.8.0.jar:2.8.0]
Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot handle message; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [org.apache.kafka.clients.consumer.ConsumerRecords] for GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}], failedMessage=GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}]
	at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:352) ~[spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:92) ~[spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:53) ~[spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2586) [spring-kafka-2.8.0.jar:2.8.0]
	... 12 common frames omitted
Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [org.apache.kafka.clients.consumer.ConsumerRecords] for GenericMessage [payload={"index":"1000","ip":"A6-B1-C1-D6-FE-34","timestamp":1638263084,"type":"6","value":[]}, headers={kafka_offset=71, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@585c68b4, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedMessageKey=key, kafka_receivedTopic=heartbeat4, kafka_receivedTimestamp=1638495075056, kafka_acknowledgment=Acknowledgment for heartbeat4-0@71, kafka_groupId=gateway2}]
	at org.springframework.messaging.handler.annotation.support.PayloadMethodArgumentResolver.resolveArgument(PayloadMethodArgumentResolver.java:145) ~[spring-messaging-5.3.13.jar:5.3.13]
	at org.springframework.kafka.annotation.KafkaNullAwarePayloadArgumentResolver.resolveArgument(KafkaNullAwarePayloadArgumentResolver.java:46) ~[spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:117) ~[spring-messaging-5.3.13.jar:5.3.13]
	at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:147) ~[spring-messaging-5.3.13.jar:5.3.13]
	at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:115) ~[spring-messaging-5.3.13.jar:5.3.13]
	at org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:56) ~[spring-kafka-2.8.0.jar:2.8.0]
	at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:347) ~[spring-kafka-2.8.0.jar:2.8.0]
	... 15 common frames omitted

我的消费端写法如下：

@KafkaListener(topics = "heartbeat4", groupId = "gateway2")
    public void listensGroupPro(ConsumerRecords<String, String> records, Acknowledgment ack) {

当然我最一开始肯定不是这么简单的写法，只不过后面这个bug，我逐步去掉了一些配置。

我看报错是类型转换异常，我就又换成了

@KafkaListener(topics = "heartbeat4", groupId = "gateway2")
    public void listensGroupPro(List<ConsumerRecord<String, String>> records, Acknowledgment ack) {
        System.out.println(records);
        for (ConsumerRecord<String, String> record : records) {
            System.out.println(record.value());
        }
        // 手动提交offset，里面的逻辑是采用的同步提交，尝试3次
        ack.acknowledge();
    }

可是还是报错，是在for循环里面报的错，并且离谱的是sout可以正常打印，但是打断点发现我的字符串竟然被拆分了！

 

 这显然不对啊，我后面又尝试把String换成？，还是不行，debug里面的源码也是找的迷路，看了很多博客，都是正常使用，其实我最一开始也是这样没问题的，不过昨天一重启批量消费就不行了！后来看很多博客配置，他们都是在yml里面正常配置，然后去自己手写一些工厂类读取配置然后注入容器。

factory.setBatchListener(batchListener);

我觉得这样的话，springkafka出的就没有意义了。后来果然让我找到了这个配置

spring.kafka.listener.type=batch

而默认是single的！

我也不知道为什么要这样默认，不过改掉之后就可以批量消费了。最不能理解的是为什么一开始我没有加却可以批量消费正常使用，然后突然拉闸。

生产端代码：

@RestController
@RequestMapping("/mmm")
public class TestProducer {
    @Autowired
    private KafkaTemplate<String,String> kafkaTemplate;

    @GetMapping(value = "/aaa")
    public  void aaa(){
        for (int i = 0; i < 10 ; i++) {
            kafkaTemplate.send("heartbeat4", "key", "{\"index\":\"1000\",\"ip\":\"666666666666666\",\"timestamp\":1638263084,\"type\":\"6\",\"value\":[]}");
            System.out.println("发送成功");
        }
    }
}

完整配置：

spring:
  kafka:
    bootstrap-servers: 你的server
    producer:
      retries: 3
      batch-size: 16384
      buffer-memory: 33554432
      acks: 1
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
    consumer:
      group-id: default-group
      enable-auto-commit: false
      auto-offset-reset: latest
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      max-poll-records: 500

    listener:
      # 当每⼀条记录被消费者监听器（ListenerConsumer）处理之后提交
      # RECORD
      # 当每⼀批poll()的数据被消费者监听器（ListenerConsumer）处理之后提交
      # BATCH
      # 当每⼀批poll()的数据被消费者监听器（ListenerConsumer）处理之后，距离上
      # 次提交时间⼤于TIME时提交
      # TIME
      # 当每⼀批poll()的数据被消费者监听器（ListenerConsumer）处理之后，被处理
      # record数量⼤于等于COUNT时提交
      # COUNT
      # TIME | COUNT　有⼀个条件满⾜时提交
      # COUNT_TIME
      # 当每⼀批poll()的数据被消费者监听器（ListenerConsumer）处理之后, ⼿动调
      # ⽤Acknowledgment.acknowledge()后提交
      # MANUAL
      # ⼿动调⽤Acknowledgment.acknowledge()后⽴即提交，⼀般使⽤这种
      # MANUAL_IMMEDIATE
      ack-mode: MANUAL_IMMEDIATE
      type: batch

如有疑问可以评论区提问。

如需转载，请务必声明原处。

【异常】ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
本本本添哥
09-09 423
这个异常常见于使用 Spring 框架的消息驱动编程模式中，例如使用 Spring Integration 或 Spring AMQP（用于处理 RabbitMQ 消息）时。检查方法内部是否有任何未捕获的异常，并确保所有必要的异常都得到了妥善处理。是一个在消息监听器（listener）尝试处理消息时抛出的异常，通常这意味着监听器方法在执行过程中遇到了问题。：确保监听器方法的参数类型与传入的消息类型匹配。：如果监听器方法依赖于某些 Bean，确保这些 Bean 已经正确地配置并可以在监听器方法中使用。
详细解决redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
念兮为美
03-18 1万+
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool的多种详细的解决方法
java kafka批量消费设置_Springboot注解@KafkaListener实现Kafka批量消费
weixin_30345333的博客
03-02 2253
在使用时Kafka时，经常遇到大批量消息在队列中，如果一个消息一个消息的消费的话效率太低下了，所以批量消费消息是很有必要的，废话不多数，直接上代码。批量监听器从版本1.1开始，@KafkaListener可以被配置为批量接收从Kafka话题队列中的Message。要配置监听器容器工厂以创建批处理侦听器，需要设置batchListener属性为true，代码如下：@BeanKafkaListener...
使用Spring-kafka的@KafkaListener报错信息Listener method could not be invoked with the incoming message
CJJ的博客
05-19 1万+
使用Spring-kafka的@KafkaListener报错信息Listener method could not be invoked with the incoming message问题背景报错原因解决办法 问题背景 先说一下背景，我在写一个kafka消费项目的时候，为了方便就用的spring-kafka依赖包，启动项目报了这个错，百度好几遍都没找到解决办法，后面再stack overflow找到答案了，在此记录一下，错误如下： 我个人觉得这个错误信息很关键： Listener method c
kafka问题之Listener method could not be invoked with the incoming message
高新富的博客
11-11 1万+
文章目录1.问题描述 1.问题描述 2020-11-11 16:18:20.180 ERROR 23834 --- [ntainer#1-0-C-1] o.s.kafka.listener.LoggingErrorHandler : Error while processing: ConsumerRecord(topic = sync_product, partition = 0, leaderEpoch = 0, offset = 0, CreateTime = 1605082700035, ser
Listener method could not be invoked with the incoming messageEndpoint handler details:Method
dayejia12的博客
06-13 3201
Listener method could not be invoked with the incoming messageEndpoint handler details:Method
rabbitmq：Listener method could not be invoked with the incoming message
qq_41992429的博客
01-10 1301
rabbitmq报错：Listener method could not be invoked with the incoming message
springboot消费kafka Listener method could not be invoked with the incoming message
zzh3864的博客
03-23 4528
bug背景：使用mysql cdc kafka 处理消息，当mysql删除一条记录时会触发此bug 错误信息： org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message Endpoint handler details: Method [public void com.flex.notify.listen
SQL Developer连接本地Oracle失败（ Listener refused the connection with...)(The Network Adapter could not..)
iwlnner的博客
12-09 4205
今天在公司电脑安装了Oracle使用sql developer测试连接的时候，提示错误： 状态: 失败 -测试失败: Listener refused the connection with the following error: ORA-12505, TNS:listener does not currently know of SID given in connect descriptor 百度了一下，说是oracle安装路径下的tnsnames.ora文件的service_name(即sid)配
ORA-12154: TNS:could not resolve the connect identifier specified
热门推荐
天天开心鸭！
11-27 2万+
前言：   在项目中需要连接数据库进行查询、插入、更新等操作，但是在运行程序的时候报了无法连接数据库的异常。   项目环境：   VisualStudio2008； Oracle11_2g； Windows8.1；   问题原因：   1.未设置好环境变量 2.Vs与Oracle连接异常 3.代码中设置登录账号密码解析错误 4.Oracle数据库与PL/SQL设置问题...
【jms】Listener method could not be invoked with incoming message
当白的技术小窝 ( git : https://github.com/canwhite )
08-30 464
引入错误导致的问题
Listener method could not be invoked with the incoming message和 Cannot convert from [java.lang.Strin
天道酬勤
07-03 1万+
记录一下入门RabbitMQ问题 问题： org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message 和 org.springframework.messaging...
ListenerExecutionFailedException: Listener method could not be invoked with the incoming message End
weixin_41872664的博客
09-14 2159
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message Endpoint handler details: 1.检查生产者和消费者的参数类型是否相同 2.检查序列化机制是否一致 ...
rabbitmq报错：Listener method could not be invoked with the incoming message
qq_41992429的博客
01-09 813
rabbitmq报错：Listener method could not be invoked with the incoming message
Listener method could not be invoked with the incoming message rabbitmq监听失败
HECHANGLONG421的博客
06-06 8935
很大一部分原因是由于你生产者传消息给mq的类型与你消费者接受的类型不一致 比如你传json去队列里面 结果使用对象去接收肯定报错
记ribbon传递消息中出现的问题，Listener method could not be invoked with the incoming message rabbitmq
weixin_43858776的博客
08-08 1870
有两种情况 1，你引入的包错了。 2，很大一部分原因是由于你生产者传消息给mq的类型与你消费者接受的类型不一致 比如我本来传递的对象，接收对象，但是传递的对象加了双引号，变为字符串，就出现错误 ...
Listener method could not be invoked with the incoming message消息队列RabbitMQ项目启动报错及监听队列报错
孟德尚且敬我，汝意欲何为
05-07 1863
Listener method could not be invoked with the incoming message 报错如图： 说是不能调用监听器的方法，问题原因是Channel依赖导错 应该是接口的rabbit.client，而不是方法的AMQP.Channel，同时注意message的类型，amqp.core.Message ...
RabbitMQ Listener method could not be invoked with the incoming message错误
最新发布
weixin_44922423的博客
07-04 381
该错误有很多不同类型的问题，我遇到的问题为生产者发送的消息为map类型，消费者入参为String。
org.springframework.jms.listener.adapter.ListenerExecutionFailedException MQ监听错误
weixin_44185736的博客
05-11 4535
文章目录错误提示：错误原因：解决思路： 错误提示： org.springframework.jms.listener.adapter.ListenerExecutionFailedException: Listener method could not be invoked with incoming message 错误原因： 发送消息时的数据格式和接收消息的类型不一致，就会导致该错误。 比如:ActiveMq中有两种消息的数据结构，TextMessage和MapMessage。 下面图1和图2体现了一
ORA-12516, TNS:listener could not find available handler with matching protocol stack
03-29
This error message typically indicates that the Oracle listener process is unable to establish a connection to a database instance. There may be several causes for this issue, including: 1. ...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值