storm 使用过程中遇到的问题

最新推荐文章于 2018-08-19 23:11:22 发布

转载最新推荐文章于 2018-08-19 23:11:22 发布 · 4.5k 阅读

storm 专栏收录该内容

3 篇文章

订阅专栏

本文解决了一个关于Apache Storm中Executor频繁重启的问题。原因是Worker进程内存不足导致Executor无法正常运行。通过增加分配给Worker的内存，成功解决了问题。

1 bolt不停重启，excutor无法启动。

nimbus日志类似如下（），

2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[34
34] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[4 4]
not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[40
40] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[10
10] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[16
16] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[22
22] not alive
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Executor MITAS3-74-1394565794:[28
28] not alive

2014-03-12 10:55:06 b.s.s.EvenScheduler [INFO] Available slots:
(["5d105f66-1add-421b-8265-e7340a95928c" 6700]
["32ab1745-c260-4491-ae4d-92dcc5d14a62" 6700])
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Reassigning MITAS3-74-1394565794 to
6 slots
2014-03-12 10:55:06 b.s.d.nimbus [INFO] Reassign executors: [[34 34] [4 4]
[40 40] [10 10] [16 16] [22 22] [28 28]]

原因是：work的内存设置太小，导致excutor无法启动

更改方法修改storm work配置，增大内存。

## to worker 
worker.childopts: "-Xmx4096m"

参考：

I have encountered the same issue in a case of out of memory in worker
process. Try increase the memory of the wokers by setting nimbus.childopts
property. Also, if you are creating short living object at higher rate
use +UseG1GC
. Since you are saying that you hold data in your memory, I'm suspecting
(as I said) an OutOfMemeory error. Don't cover this just by increasing the
heap size, but also I recommend to profile your worker and see if you have
a memory leak.
转自：http://qnalist.com/questions/4769895/topology-restart-due-to-executor-not-alive