Storm Worker端口冲突

最新推荐文章于 2025-08-29 01:00:05 发布

原创最新推荐文章于 2025-08-29 01:00:05 发布 · 816 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#大数据

Storm 专栏收录该内容

25 篇文章

订阅专栏

本文介绍了当Storm任务启动过程中遇到Worker端口被占用的问题时的解决方案。通过调整内核参数，预留特定端口供Worker使用，避免与其他服务冲突。

storm任务启动过程中一个worker启动失败，通过supervisor日志发现一直在尝试启动，最后定位到时worker的端口被其他进程占用了导致启动失败，尤其是当机器上有大量短连接时会增加worker端口被占的几率。

解决办法

其他服务避开worker端口
通过内核参数配置将storm使用到的端口配置成以监听的方式启动

三种查看随机端口范围

sysctl  net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 1024     65000


cat /etc/sysctl.conf |grep local
net.ipv4.ip_local_port_range = 1024 65000


cat /proc/sys/net/ipv4/ip_local_port_range
1024    65000

如果storm使用的端口在这个范围内就有冲突的几率。

配置预留端口

echo "5710-5714,15710-15714" > /proc/sys/net/ipv4/ip_local_reserved_ports


shell> vim /etc/sysctl.conf
net.ipv4.ip_local_reserved_ports = 5710-5714,15710-15714
shell> sysctl -p

net.ipv4.ip_local_reserved_ports描述

ip_local_reserved_ports - list of comma separated ranges Specify the ports which are reserved for known third-party applications. 

These ports will not be used by automatic port assignments (e.g. when calling connect() or bind() with port number 0). Explicit port allocation behavior is unchanged.

The format used for both input and output is a comma separated list of ranges (e.g. "1,2-4,10-10" for ports 1, 2, 3, 4 and 10). Writing to the file will clear all previously reserved ports and update the current list with the one given in the input.

Note that ip_local_port_range and ip_local_reserved_ports settings are independent and both are considered by the kernel when determining which ports are available for automatic port assignments.

You can reserve ports which are not in the current
ip_local_port_range, e.g.:
$ cat /proc/sys/net/ipv4/ip_local_port_range 32000        61000
$ cat /proc/sys/net/ipv4/ip_local_reserved_ports 8080,9148
although this is redundant. However such a setting is useful if later the port range is changed to a value that will include the reserved ports.
Default: Empty