ES 3节点部署step by step

前置任务

在开始部署前需要先下载一下elasticsearch&ik分词器

1. 下载
   下载7.16 版本的es(因为7.1x是最后一个支持jdk1.8的es) 解压后放到/opt（我一般习惯放到此目录）目录下
   elasticsearch-7.16.3-linux-x86_64.tar.gz
   为了更好的支持中文，需要下载ik分词器
   elasticsearch-analysis-ik-7.16.3.zip
2. 创建用户名为es

adduser es
passwd es
testpassword
chown -R es:es elasticsearch-7.16.3/
chmod 770 elasticsearch-7.16.3/

3. 设置es用户的process数量
   vim /etc/security/limits.conf

es hard nofile 65536
es soft nofile 65536

4. 设置es用户的max number of threads
   vim /etc/security/limits.d/90-nproc.conf

*          soft    nproc     4096
root       soft    nproc     unlimited

5. 设置es用户的最大虚拟内存
   vim /etc/sysctl.conf

# Controls max virtual memory areas
vm.max_map_count=262144

6. 刷新配置

sysctl -p

部署集群

以下是进入elasticsearch-7.16.3目录后的操作，当然elasticsearch-7.16.3是要在es账号下有权限的。
假设有192.168.1.1、192.168.1.2、192.168.1.3三台服务器，以下步骤要在三台集群上都要配置

1. 创建data目录

mkdir data

2. 进入config目录下创建证书目录

mkdir certificate

3. 修改elasticsearch.yml文件(可参考)

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: common-server
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: common-server-3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-7.16.3/data
#
# Path to log files:
#
path.logs: /opt/elasticsearch-7.16.3/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 192.168.1.3
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200

transport.tcp.port: 9300
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.1.1:9300", "192.168.1.2:9300", "192.168.1.3:9300"]
## 每隔多长时间ping一个node
##master选举/节点间通讯超时时间(这个时间的配置需要根据实际情况设置)
discovery.zen.fd.ping_interval: 30s

## 每次ping的超时时间
discovery.zen.fd.ping_timeout: 60s

## 一个node被ping多少次失败就认为是故障了
discovery.zen.fd.ping_retries: 6
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["common-server-1"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security ----------------------------------
#
#                                 *** WARNING ***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don’t have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features. 
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html

##集群脑裂问题参数配置
## elasticsearch则可以配置返回消息的节点数量, 一般情况下会配置(n/2 + 1)个节点
discovery.zen.minimum_master_nodes: 2
## 多少个节点启动后就可以组成集群
gateway.recover_after_nodes: 2
## 期望多少个节点联通之后才开始分配shard
gateway.expected_nodes: 3
## 超时时间
gateway.recover_after_time: 1m
node.master: true
node.data: true
node.ingest: true

4. 切换至es账号
5. 通过./elasticsearch -d命令启动es集群检测是否有异常
   正常是可以启动的，并且通过curl -L -X GET 'http://192.168.1.1:9200/_cat/nodes?v’可以看到集群状态

ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
192.168.1.2           62          97  11    0.38    0.22     0.13 cdfhilmrstw -      common-server-2
192.168.1.1           35          98  23    0.57    0.31     0.22 cdfhilmrstw *      common-server-1
192.168.1.3           69          98  18    0.20    0.39     0.44 cdfhilmrstw -      common-server-3

生成ca证书打开用户认证

1. 切换至es用户
2. es的bin目录下执行./elasticsearch-certutil ca，然后两次回车
3. es的bin目录下执行./elasticsearch-certutil cert --ca elastic-stack-ca.p12，然后三次回车
4. 将生成的elastic-certificates.p12、elastic-stack-ca.p12用es用户copy到es的config/certificate目录下面
5. 修改三台服务器的elasticsearch.yml文件增加如下配置

# 跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Type,Content-Length

xpack.security.enabled: true
xpack.security.authc.accept_default_password: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /opt/elasticsearch-7.16.3/config/certificate/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /opt/elasticsearch-7.16.3/config/certificate/elastic-certificates.p12

6. 重启es三集群，查看服务状态是否异常
7. 在其中一台服务器上设置es标准账号的密码，在es的bin目录下执行

./elasticsearch-setup-passwords interactive

安装IK中文分词器

1. 使用es账户将elasticsearch-analysis-ik-7.16.3.zip上传至es用户的用户目录（/home/es）下
2. 解压至当前目录下unzip elasticsearch-analysis-ik-7.16.3.zip -d ./elasticsearch-analysis-ik-7.16.3
3. 将解压后的文件拷贝至es的plugin目录下 cp -r elasticsearch-analysis-ik-7.16.3 /opt/elasticsearch-7.16.3/plugins
4. 重启es（先ps -ef | grep elastic找到进程id， 然后kill -9结束， 最后./elasticsearch -d重启es）
5. 通过./elasticsearch-plugin list 查看是否有安装的elasticsearch-analysis-ik-7.16.3分词器

参考资料

1. https://blog.youkuaiyun.com/chenshm/article/details/124264078
2. https://blog.youkuaiyun.com/aotumemedazhao1996/article/details/108746616
3. https://blog.youkuaiyun.com/aotumemedazhao1996/article/details/108746616