docker swarm集群下部署elasticsearch7.6.2集群+kibana7.6.2+es-head+中文分词

[root@elcndc2zhda01 ~]# docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
skq2d634av8scq5uumciwuedf *   elcndc2zhda01       Ready               Active              Leader              19.03.8
1j1i345greywliwuyuwro01if     elcndc2zhda02       Ready               Active              Reachable           19.03.8
xqxl9gs9igb7jyo3cohsarehi     elcndc2zhda03       Ready               Active              Reachable           19.03.8

注意：注意：为了方便我三个集群节点已经配置了nas共享挂载目录，为/data目录。如果大家没有配置共享目录，那有些文件需要三个服务器都创建，比如下边提到的kibana。所以下边只要提到/data目录下的文件，我这里是三台服务器共享的，操作一个服务器即可，如果没有共享目录，需要每个服务器单独操作。

准备镜像

三台服务器分别下载官方镜像：elasticsearch:7.6.2,kibana:7.6.2(需要与es版本一致),elasticsearch-head:5

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2
docker pull docker.elastic.co/kibana/kibana:7.6.2
docker pull mobz/elasticsearch-head:5

镜像下载好后，开始准备配置es

配置elasticsearch

在/data下为是三个服务器分别创建配置文件存放目录

mkdir /data/erms/es/{kinana,node1,node2,node3} -p

三个node分别是给三个服务器用的，如果没有共享目录也可以手动一个服务器创建一个node目录。

创建后效果：

[root@elcndc2zhda01 es]# cd /data/erms/es
[root@elcndc2zhda01 es]# ls
kinana  node1  node2  node3
[root@elcndc2zhda01 es]#

添加elasticsearch配置文件

在node1目录下新建elasticsearch.yml配置文件内容如下：

将配置文件中的 discovery.seed_hosts内容替换为自己集群的ip

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
 cluster.name: es-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
 node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
 network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
 http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
 discovery.seed_hosts: ["10.**.**.68:10010","10.**.**.69:10010","10.**.**.70:10010"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
 cluster.initial_master_nodes: ["node-1","node-2","node-3"]
 
# bootstrap.memory_lock: true
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
 http.cors.enabled: true
 http.cors.allow-origin: '*'
 http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

在node2目录下新建elasticsearch.yml配置文件内容如下：

将配置文件中的 discovery.seed_hosts内容替换为自己集群的ip

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
 cluster.name: es-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
 node.name: node-2
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
 network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
 http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
 discovery.seed_hosts: ["10.**.**.68:10010","10.**.**.69:10010","10.**.**.70:10010"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
 cluster.initial_master_nodes: ["node-1","node-2","node-3"]
 
# bootstrap.memory_lock: true
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
 http.cors.enabled: true
 http.cors.allow-origin: '*'
 http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

在node3目录下新建elasticsearch.yml配置文件内容如下：

将配置文件中的 discovery.seed_hosts内容替换为自己集群的ip

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
 cluster.name: es-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
 node.name: node-3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
 network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
 http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
 discovery.seed_hosts: ["10.**.**.68:10010","10.**.**.69:10010","10.**.**.70:10010"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
 cluster.initial_master_nodes: ["node-1","node-2","node-3"]
 
# bootstrap.memory_lock: true
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
 http.cors.enabled: true
 http.cors.allow-origin: '*'
 http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

添加Dockerfile文件

向node1，node2，node3三个目录下均添加Dockerfile，内容如下：

这里第三行是为了添加中文分词，如果不需要可删掉。

FROM docker.elastic.co/elasticsearch/elasticsearch:7.6.2
COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/
ADD ik /usr/share/elasticsearch/plugins/ik
ADD ik/config /data/erms/es/node1/ik/config

添加中文分词器

分词器下载网址：https://github.com/medcl/elasticsearch-analysis-ik/releases

选择自己es对应的版本，这里下载的7.6.2：https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.6.2

将elasticsearch-analysis-ik-7.6.2.zip 上传到ik目录下，

解压elasticsearch-analysis-ik-7.6.2.zip

删除elasticsearch-analysis-ik-7.6.2.zip

命令依次如下：

[root@elcndc2zhda01 node1]# mkdir ik
[root@elcndc2zhda01 node1]# cd ik
[root@elcndc2zhda01 ik]# unzip elasticsearch-analysis-ik-7.6.2.zip 
Archive:  elasticsearch-analysis-ik-7.6.2.zip
   creating: config/
  inflating: config/main.dic         
  inflating: config/quantifier.dic   
  inflating: config/extra_single_word_full.dic  
  inflating: config/IKAnalyzer.cfg.xml  
  inflating: config/surname.dic      
  inflating: config/suffix.dic       
  inflating: config/stopword.dic     
  inflating: config/extra_main.dic   
  inflating: config/extra_stopword.dic  
  inflating: config/preposition.dic  
  inflating: config/extra_single_word_low_freq.dic  
  inflating: config/extra_single_word.dic  
  inflating: elasticsearch-analysis-ik-7.6.2.jar  
  inflating: httpclient-4.5.2.jar    
  inflating: httpcore-4.4.4.jar      
  inflating: commons-logging-1.2.jar  
  inflating: commons-codec-1.9.jar   
  inflating: plugin-descriptor.properties  
  inflating: plugin-security.policy  
[root@elcndc2zhda01 ik]# rm -rf elasticsearch-analysis-ik-7.6.2.zip 
[root@elcndc2zhda01 ik]# ls
commons-codec-1.9.jar    config                               httpclient-4.5.2.jar  plugin-descriptor.properties
commons-logging-1.2.jar  elasticsearch-analysis-ik-7.6.2.jar  httpcore-4.4.4.jar    plugin-security.policy

启动elasticsearch

打包elasticsearch镜像

三台服务器需要分别打包镜像，这里因为es主从节点配置有点差异所以三个节点的es配置文件有所不同，才这么操作。有更好方式的请给我留言。不过我这个方式集群运行效果还不错。

68服务器下执行：

cd /data/erms/es/node1
docker build -t erms/es .

69服务器下执行：

cd /data/erms/es/node2
docker build -t erms/es .

70服务器下执行：

cd /data/erms/es/node3
docker build -t erms/es .

编写compose文件

三台服务器分别对应的docker compose文件如下：

注意：这里es索引数据的持久化储存，没有单独创建目录存放，而是通过docker的volumes，由docker自行管理的。

这里向外暴露的端口号是10010

docker-compose-esnode1.yml

version: '3.3'

services:
  ermsesnode1:
      hostname: ermsesnode1
      image: erms/es
      labels:
        "type": "2"
      networks:
        - edoc2_default
      ports:
      - target: 9200
        published: 10010
        mode: host
      environment:
      - node.name=ermsesnode1
      - discovery.seed_hosts=ermsesnode2,ermsesnode3
      - cluster.initial_master_nodes=ermsesnode1,ermsesnode2,ermsesnode3
      - cluster.name=es-cluster
      #- bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      volumes:
      - esdata2:/usr/share/elasticsearch/data
      deploy:
        #placement:
        #  constraints:
        #   - node.labels.nodetype == InDrive
        replicas: 1
        restart_policy:
          condition: on-failure
volumes:
  esdata2:
    driver: local
networks:
  edoc2_default:
     external:
       name: macrowing

docker-compose-esnode2.yml

version: '3.3'

services:
  ermsesnode2:
      hostname: ermsesnode2
      image: erms/es
      labels:
        "type": "2"
      networks:
        - edoc2_default
      ports:
      - target: 9200
        published: 10010
        mode: host
      environment:
      - node.name=ermsesnode2
      - discovery.seed_hosts=ermsesnode1,ermsesnode3
      - cluster.initial_master_nodes=ermsesnode1,ermsesnode2,ermsesnode3
      - cluster.name=es-cluster
      #- bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      volumes:
      - esdata2:/usr/share/elasticsearch/data
      deploy:
        #placement:
        #  constraints:
        #   - node.labels.nodetype == InDrive
        replicas: 1
        restart_policy:
          condition: on-failure
volumes:
  esdata2:
    driver: local
networks:
  edoc2_default:
     external:
       name: macrowing

docker-compose-esnode3.yml

version: '3.3'

services:
  ermsesnode3:
      hostname: ermsesnode3
      image: erms/es
      labels:
        "type": "2"
      networks:
        - edoc2_default
      ports:
      - target: 9200
        published: 10010
        mode: host
      environment:
      - node.name=ermsesnode3
      - discovery.seed_hosts=ermsesnode2,ermsesnode1
      - cluster.initial_master_nodes=ermsesnode1,ermsesnode2,ermsesnode3
      - cluster.name=es-cluster
      #- bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      volumes:
      - esdata2:/usr/share/elasticsearch/data
      deploy:
        #placement:
        #  constraints:
        #   - node.labels.nodetype == InDrive
        replicas: 1
        restart_policy:
          condition: on-failure
volumes:
  esdata2:
    driver: local 
networks:
  edoc2_default:
     external:
       name: macrowing

启动elasticsearch

依次执行三个compose文件，启动es三个节点。（这里是都在leader node（主节点）下执行的，每个服务器执行一个也没事儿。）

[root@elcndc2zhda01 ik]# docker stack deploy -c /data/erms/compose/docker-compose-esnode1.yml erms
Creating service erms_ermsesnode1
[root@elcndc2zhda01 ik]# docker stack deploy -c /data/erms/compose/docker-compose-esnode2.yml erms
Creating service erms_ermsesnode2
[root@elcndc2zhda01 ik]# docker stack deploy -c /data/erms/compose/docker-compose-esnode3.yml erms
Creating service erms_ermsesnode3
[root@elcndc2zhda01 ik]# docker service ls
ID                  NAME                   MODE                REPLICAS            IMAGE                                                       PORTS
cmkfhkxjya79        erms_ermsesnode1       replicated          1/1                 erms/es:latest                                              
vw0kde788sx4        erms_ermsesnode2       replicated          1/1                 erms/es:latest                                              
vrt2jf9qf6d7        erms_ermsesnode3       replicated          1/1                 erms/es:latest  
[root@elcndc2zhda01 ik]#

一般集群都会添加portainer，可以通过portainer查看一下效果如下，

说明，三个es节点分别在swarm集群的三个node上运行的。

打开浏览器输入其中一个服务器的IP:10010接口访问es如下，即证明es启动成功。

启动kibana

创建kibana配置文件

切换到开始时创建的kibana目录

cd /data/erms/es/kinana/

创建配置文件kibana.yml，内容如下

#
## ** THIS IS AN AUTO-GENERATED FILE **
##
#  
#  # Default Kibana configuration for docker target
server.name: kibana
server.host: "0"
#这里写你的es第一个node的地址
elasticsearch.hosts: [ "http://ermsesnode1:9200","http://ermsesnode1:9200","http://ermsesnode1:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true
i18n.locale: zh-CN

创建compose文件，kinana-compse.yml，这里kibana启动一个节点就够了，开放端口号为10009。内容如下：

version: '3.5'
services:
   kibana:
    image: docker.elastic.co/kibana/kibana:7.6.2
    container_name: kibana
    networks:
       - edoc2_default
    ports:
      - 10009:5601
    volumes:
      - /etc/localtime:/etc/localtime
      - /data/erms/es/kinana/kibana.yml:/usr/share/kibana/config/kibana.yml:rw
networks:
  edoc2_default:
    external: true
    name: macrowing

启动kibana

[root@elcndc2zhda01 kinana]# docker stack deploy -c /data/erms/es/kinana/kinana-compse.yml erms
[root@elcndc2zhda01 kinana]# docker service ls
ID                  NAME                   MODE                REPLICAS            IMAGE                                                       PORTS
cmkfhkxjya79        erms_ermsesnode1       replicated          1/1                 erms/es:latest                                              
vw0kde788sx4        erms_ermsesnode2       replicated          1/1                 erms/es:latest                                              
vrt2jf9qf6d7        erms_ermsesnode3       replicated          1/1                 erms/es:latest                                              
2gi11fjxl67k        erms_kibana            replicated          1/1                 docker.elastic.co/kibana/kibana:7.6.2                       *:10009->5601/tcp

portainer：

输入集群任意ip:10009连接kibana，第一次进入需要选择，选第二项即可进入一下界面，点击小扳手进入开发工具界面

kibana也有监测等实用功能。如果需要安装es head的话，往下看。

启动elasticsearch-head

1、单机运行head

直接docker启动，例如我们在69服务器上执行docker命令，暴露端口号为10001

docker run -d -p 10001:9100 docker.io/mobz/elasticsearch-head:5

启动成功后打开浏览器查看69服务器的ip:10001访问head，输入es集群地址点击链接，可以看到集群健康状态、索引以及es和kibana相关信息。

2、集群模式启动

创建eshead-compse.yml

version: '3.3'
services:
  elasticsearch-head:
    image: mobz/elasticsearch-head:5
    container_name: elasticsearch-head       
    ports:
       - "10001:9100"           
    links:
       - ermsesnode1

放到集群中启动，

docker stack deploy -c /data/erms/compose/eshead-compse.yml erms

查看portainer

这样，head也被添加到集群中进行管理了。方便维护，并且输入集群任意ip都可以访问到。通过集群域名访问也可以。

使用方式与单机启动一样。

几篇关于elasticsearch的文章：

Java+springboot+elasticsearch7.6.2实现分组查询（平级+层级）并实现script多字段拼接查询以及大于小于等条件关系查询的实现

linux Centos7 安裝搭建elasticsearch7.6.2+kibana7.6.2+中文分词7.6.2详解

docker swarm集群下部署elasticsearch7.6.2集群+kibana7.6.2+es-head+中文分词