sparkstreaming整合kafka运行报错

最新推荐文章于 2023-02-21 11:26:01 发布

flashtown

最新推荐文章于 2023-02-21 11:26:01 发布

阅读量939

点赞数

CC 4.0 BY-SA版权

分类专栏：报错

本文链接：https://blog.youkuaiyun.com/flashtown/article/details/110387688

在尝试将SparkStreaming与Kafka整合时，遇到了'java.lang.NoSuchMethodError'异常，具体错误指向io.netty.buffer.PooledByteBufAllocator.defaultNumHeapArena()I方法。错误出现在Spark的NettyRpcEnv初始化过程中，可能与Netty版本冲突或依赖库不兼容有关。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

20/11/30 14:24:47 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(flashtown); groups with view permissions: Set(); users with modify permissions: Set(flashtown); groups with modify permissions: Set()
Exception in thread "main" java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.defaultNumHeapArena()I
   at org.apache.spark.network.util.NettyUtils.createPooledByteBufAllocator(NettyUtils.java:113)
   at org.apache.spark.network.client.TransportClientFactory.<init>(TransportClientFactory.java:106)
   at org.apache.spark.network.TransportContext.createClientFactory(TransportContext.java:99)
   at org.apache.spark.rpc.netty.NettyRpcEnv.<init>(NettyRpcEnv.scala:71)
   at org.apache.spark.rpc.netty.NettyRpcEnvFactory.create(NettyRpcEnv.scala:461)
   at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:57)
   at org.apache.spark.Spark

最低0.47元/天解锁文章