发现有2台服务器4月中旬就要到期了,上面的业务量属实挺小的,没有续费必要。直接合并到其他节点可以节省成本
把lv2节点的elasticsearch迁移到新节点lv7,发现无法加入集群了
iCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:19,668][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:22,782][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:25,873][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:26,592][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:28,760][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:28,875][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:28,970][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:29,513][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:31,345][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:32,071][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:32,295][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:33,077][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:33,928][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:34,038][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:35,176][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
[2025-04-02T20:01:35,377][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:36,514][INFO ][o.e.x.s.a.AuthenticationService] [node-1] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2025-04-02T20:01:38,276][INFO ][o.e.d.z.ZenDiscovery ] [node-1] failed to send join request to master [{node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-1][172.18.142.49:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[node-1][172.17.0.1:9300] handshake failed. unexpected remote node {node-1}{Lehwm7ZHSNiCYdhj0R-m0A}{ccMEAELgTWu9pvE1fTH2Og}{172.18.142.49}{172.18.142.49:9300}{ml.machine_memory=16544047104, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}]; ]
1.先看看配置,除了node-name不一样外,3个节点的配置都是一样的
[root@lv7 elasticsearch-6.8.11]# yy config/elasticsearch.yml
cluster.name: lv-cluster
node.name: node-1
node.master: true
node.data: true
network.host: 0.0.0.0
http.port: 9201
discovery.zen.ping.unicast.hosts: ["lv2", "lv8", "lv9"]
discovery.zen.minimum_master_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.truststore.path: /data1/lv/elastic-stack/elasticsearch-6.8.11/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.keystore.path: /data1/lv/elastic-stack/elasticsearch-6.8.11/config/certs/elastic-certificates.p12
存在了一个问题:
- 节点自环问题
- 配置中
network.host: 0.0.0.0
可能导致节点误认自身IP - 日志显示节点尝试加入自己(
172.18.142.49:9300
)
- 配置中
之前的运维还是比较偷懒的,后面把配置文件调整为。
[root@lv7 config]# yy elasticsearch.yml
cluster.name: lv-cluster
node.name: node-1
node.master: false
node.data: true
network.host: 172.18.142.48
transport.tcp.port: 9300
http.port: 9201
discovery.zen.ping.unicast.hosts: ["172.18.142.49:9300", "172.18.142.50:9300"]
discovery.zen.minimum_master_nodes: 2
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /data1/lv/elastic-stack/elasticsearch-6.8.11/config/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /data1/lv/elastic-stack/elasticsearch-6.8.11/config/certs/elastic-certificates.p12
# 明确绑定IP,禁止使用0.0.0.0
network.host: 172.18.142.49 # lv7的真实IP
transport.tcp.port: 9300 # 确保集群通信端口一致
# 使用IP而非主机名(避免DNS问题)
discovery.zen.ping.unicast.hosts: ["172.18.142.50:9300", "172.18.142.51:9300"] # lv8/lv9的IP
# lv7不作为主节点,应为现在的es master是lv8
node.master: false
node.data: true
discovery.zen.minimum_master_nodes: 2 # 总master节点数/2 + 1
最后加入成功
最后加入成功