org.apache.storm.thrift.transport.TTransportException

Exception in thread "main" java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:256)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:311)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
at com.yiguan.topology.RtAppTopology.run(RtAppTopology.java:93)
at com.yiguan.topology.RtStart.control(RtStart.java:34)
at com.yiguan.topology.RtStart.main(RtStart.java:18)
Caused by: org.apache.storm.thrift.transport.TTransportException
at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:294)
at org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:278)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:244)

... 5 more


或者:

org.apache.thrift.transport.TTransportException: Frame size (17302738) larger than max length (16384000)



 这个问题的原因是序列化后的topology对象过大导致的,通常可能是你在spout/bolt中创建了一个大对象(比如bitmap, 大数组等),导致序列化后对象的大小超过了thrift的max frame size(thrift中16384000这个值是写死的,只能调小不能调大)。在JStorm中,如果需要在spout/bolt中创建大对象,建议是在open/prepare方法中来做,延迟对象的创建时间。参见:https://github.com/alibaba/jstorm/issues/230


在提交topology到集群之前是不能做大量数据加载。





评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

_小海_

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值