【Kafka】一台服务器启动多个kafka实例

原创已于 2022-03-30 12:15:00 修改 · 5.2k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#Kafka

于 2022-03-30 12:07:08 首次发布

Kafka 专栏收录该内容

5 篇文章

订阅专栏

本文介绍在本地虚拟机中启动两个Kafka实例时遇到的端口冲突问题及解决方案。通过调整JMX端口号成功启动第二个实例。

在学习和研究 Kafka 的过程中，想要在本地虚拟机里面启动两个 Kafka 实例，发现第二个实例总是启动失败，而网上的部分文章，想当然的克隆出一个server.properties，直接启动即可。

为了不让学习 Kafka 的小伙伴们掉到坑里面，这里给大家描述下我踩坑的过程，以及如何从坑里面爬出来的。

第一个 Kafka 实例的配置：

# server.properties
broker.id=0
listeners=PLAINTEXT://192.168.223.200:9092
advertised.listeners=PLAINTEXT://192.168.223.200:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/root/kafka/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.flush.interval.messages=10000
log.flush.interval.ms=1000
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.223.200:2181
zookeeper.connection.timeout.ms=60000
group.initial.rebalance.delay.ms=0

第二个 Kafka 实例的配置：

# server-1.properties
broker.id=1
listeners=PLAINTEXT://192.168.223.200:9093
advertised.listeners=PLAINTEXT://192.168.223.200:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/root/kafka/logs/instance1
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.flush.interval.messages=10000
log.flush.interval.ms=1000
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.223.200:2181
zookeeper.connection.timeout.ms=60000
group.initial.rebalance.delay.ms=0

配置对比截图：

第一个 Kafkas实例的启动成功

# /root/kafka/bin/kafka-server-start.sh -daemon /root/kafka/config/server.properties

第二个 Kafkas实例的启动失败

#  /root/kafka/bin/kafka-server-start.sh -daemon /root/kafka/config/server-1.properties

看看第二个实例启动失败的报错

Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 9010; nested exception is: 
        java.net.BindException: Address already in use (Bind failed)
sun.management.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 9010; nested exception is: 
        java.net.BindException: Address already in use (Bind failed)
        at sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:480)
        at sun.management.Agent.startAgent(Agent.java:262)
        at sun.management.Agent.startAgent(Agent.java:452)
Caused by: java.rmi.server.ExportException: Port already in use: 9010; nested exception is: 
        java.net.BindException: Address already in use (Bind failed)
        at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
        at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
        at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412)
        at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
        at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:237)
        at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
        at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173)
        at sun.management.jmxremote.SingleEntryRegistry.<init>(SingleEntryRegistry.java:49)
        at sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:816)
        at sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:468)
        ... 2 more
Caused by: java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:513)
        at java.net.ServerSocket.bind(ServerSocket.java:375)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at java.net.ServerSocket.<init>(ServerSocket.java:128)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createServerSocket(RMIDirectSocketFactory.java:45)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createServerSocket(RMIMasterSocketFactory.java:345)
        at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670)
        at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:335)
        ... 11 more

可以很明显的发现是9010这个端口冲突，导致第二个实例启动失败。检查了下，发现 9010 这个端口是在kafka-server-start.sh这个脚本里面配置的，修改下这个脚本的JMX端口，再次启动第二个实例，启动成功。

修改前：
export JMX_PORT="9010"

修改后：
# export JMX_PORT="9010"
export JMX_PORT="9011"