Storm初始化时抛出NotSerializableException

关注

原创最新推荐文章于 2026-10-12 10:09:56 发布 894 阅读 AI 写同款· GEO 优化 ›

现代数据工程师必掌握的5大核心大数据框架大数据框架是构建可扩展、高可靠数据基础设施的基石。从分布式存储（HDFS）到实时消息管道（Kafka），再到批流一体计算引擎（Spark/Flink）和统一SQL层（Hive），这些技术共同定义了数据处理的底层语义与工程边界。其核心价值在于支撑PB级数据规模下的低延迟、强一致性与可运维性，广泛应用于实时推荐、用户行为分析、风控建模与电商大屏等关键场景。本文聚焦Apache Spark、Flink、Kafka、Hive和Hadoop五大工业级框架，深入解析其在真实生产环境中的关键配置、性能命门与典型故障模式，阅读详情

启动topo抛出如下异常：

Exception in thread "main" java.lang.IllegalStateException: Spout 'DetrUpdateEvent' contains a non-serializable field of type com.gdl.trolley.stormclient.TrolleySpout$StormMessageListner, which was instantiated prior to topology creation. com.gdl.trolley.stormclient.TrolleySpout$StormMessageListner should be instantiated within the prepare method of 'DetrUpdateEvent at the earliest.
    at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:138)
    at com.gdl.gdl.gdlpsr.processor.detrupdate.DetrupdateTopology.main(DetrupdateTopology.java:42)
Caused by: java.lang.RuntimeException: java.io.NotSerializableException: com.gdl.trolley.stormclient.TrolleySpout$StormMessageListner
    at backtype.storm.utils.Utils.javaSerialize(Utils.java:196)
    at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:135)
    ... 1 more
Caused by: java.io.NotSerializableException: com.gdl.trolley.stormclient.TrolleySpout$StormMessageListner
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at backtype.storm.utils.Utils.javaSerialize(Utils.java:192)
    ... 2 more

在storm和bolt和spout中不能包含非静态且在对象创建时初始化的字段。如下面代码时不被允许的。

public class TrolleySpout extends KafkaSpout {
    private static KafkaSpoutConfig.Builder builder = KafkaSpoutConfig.builderStringBytes(UpdateConfig.getInstance().getBrokerHosts()
            , UpdateConfig.getInstance().getAppid());
    private StormMessageListner stormMessageListner = new StormMessageListner();

解决方案：对于bolt，在pepare方法中完成初始化。对于spout，要么重构，要么冒险进行序列化。
实行序列化有什么危险呢？

收录于