storm和kafka集成报java.lang.ClassNotFoundException: kafka.api.OffsetRequest解决方法

解决Kafka Spout配置错误

最新推荐文章于 2024-08-29 18:42:50 发布

weixin_40137479

最新推荐文章于 2024-08-29 18:42:50 发布

阅读量1.4k

点赞数

本文介绍了解决Apache Storm Kafka Spout配置中找不到kafka.api.OffsetRequest类的问题。通过调整Maven依赖，将kafka-clients从0.8.2.1版本升级到0.11.0.0，并排除部分冲突依赖，成功解决了NoClassDefFoundError异常。

添加依赖

    <dependency>
            <groupId>org.apache.storm</groupId>
            <artifactId>storm-kafka</artifactId>
            <version>1.1.0</version>
        </dependency>1
2
3
4
5

然后报错

Exception in thread "main" java.lang.NoClassDefFoundError: kafka/api/OffsetRequest
    at org.apache.storm.kafka.KafkaConfig.<init>(KafkaConfig.java:48)
    at org.apache.storm.kafka.SpoutConfig.<init>(SpoutConfig.java:46)
    at cn.yuli.storm.wordCount.KafkaTopo.main(KafkaTopo.java:20)
Caused by: java.lang.ClassNotFoundException: kafka.api.OffsetRequest
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 3 more1
2
3
4
5
6
7
8
9
10
11
12
13

跟着代码检查了一下
在KafkaConfig下执行下面这个方法时找不到kafka.api.OffsetRequest类

 public KafkaConfig(BrokerHosts hosts, String topic) {
        this(hosts, topic, kafka.api.OffsetRequest.DefaultClientId());
    }1
2
3

然后发觉maven默认依赖的是kafka-clients-0.8.2.1.jar，把kafka客户端换成高版本并去除冲突依赖后解决异常

<dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.11</artifactId>
            <version>0.11.0.0</version>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.zookeeper</groupId>
                    <artifactId>zookeeper</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
            </exclusions>
        </dependency>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19