阿里云服务器环境部署二 ES集群+Kibana部署_阿里云服务器部署es-优快云博客

本文链接：https://blog.youkuaiyun.com/wangqiaowq/article/details/146331876

ES部署

一、准备工作

提前开放三台服务器的9200、9300端口，需要配置kibana的服务器额外开放5601端口。只需要配置一个kibana即可连接es集群。
创建es网卡
docker network create es-net

二、安装 elasticsearch 8.14
创建目录

1、docker 拉取ES镜像
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.14.1

3个节点都要拉取

3.创建挂载目录

/home/dockerinstall/es
mkdir data plugins

# 添加文件夹权限

chmod 777 data

chmod 777 plugins
3个节点都要操作

4.启动es集群

4.1 配置文件挂载启动

编写elasticsearch.yml

mkdir config

# 进入config文件加

cd config

# 编写elasticsearch.yml

vim elasticsearch.yml

注意每台服务器需要修改的参数

#集群名称
cluster.name: es-cluster
# 节点名称
node.name: node-1
# 网络地址
network.host: 0.0.0.0
network.publish_host: 172.16.108.43
# 集群节点配置
discovery.seed_hosts: ["172.16.108.43:9300","172.16.108.42:9300","172.16.108.41:9300"]
# 主节点候选
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
#
# 客户端端口
http.port: 9200
# 集群节点端口
transport.port: 9300
#
# 是否开启安全认证
xpack.security.enabled: false
xpack.security.enrollment.enabled: true
#
# 是否开启ssl
xpack.security.http.ssl:
  enabled: false
#     #keystore.path: /usr/share/elasticsearch/config/certs/http.p12
#       #truststore.path: /usr/share/elasticsearch/config/certs/http.p12
#
# 是否开启访问安全认证
xpack.security.transport.ssl:
  enabled: false
#             #verification_mode: certificate
#             #keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
#               #truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
#
#               # 跨域配置
http.cors.enabled: true
http.cors.allow-origin: "*"
http.host: 0.0.0.0

注意每台服务器需要修改的参数
node.name —— es集群节点名称，每台服务器不同
network.publist_host —— 服务器ip

scp elasticsearch.yml root@bigdata41:/home/dockerinstall/es/config/
scp elasticsearch.yml root@bigdata42:/home/dockerinstall/es/config/

下载插件：

https://release.infinilabs.com/analysis-ik/stable/elasticsearch-analysis-ik-8.14.1.zip

解压到plugins目录

unzip elasticsearch-analysis-ik-8.14.1.zip -d analysis-ik/

rm -rf elasticsearch-analysis-ik-8.14.1.zip 解压后需要删除
或 mv elasticsearch-analysis-ik-8.14.1.zip /home/dockerinstall/

3个节点都要操作

挨个节点启动es 3个节点都要操作

docker run -d --privileged=true --name es --restart=unless-stopped --network es-net -p 9200:9200 -p 9300:9300  -v /home/dockerinstall/es/data:/usr/share/elasticsearch/data  -v /home/dockerinstall/es/plugins:/usr/share/elasticsearch/plugins  -v /home/dockerinstall/es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml docker.elastic.co/elasticsearch/elasticsearch:8.14.1

docker ps 查看或容器ID 6be433b8e0ae

docker logs 6be433b8e0ae 查看具体日志

curl 172.16.108.43:9200/_cat/plugins?v=true
插件安装已成功

为避免对外暴露ES的相关端口，暂时不安装kibana，如需使用需要参考给容器启动的kibana设置用户名和密码 - 简书

给ES索引设置模板
查看模板信息

curl -X GET "http://172.16.108.43:9200/_index_template"

设置模板信息

curl -X PUT "http://172.16.108.43:9200/_index_template/content_erp_nlp_help" \
-H "Content-Type: application/json" \
-d '{
  "index_patterns": ["content_erp*"],
  "priority": 100,
  "template": {
    "settings": {
      "analysis": {
        "analyzer": {
          "my_ik_analyzer": {
            "type": "ik_smart"
          }
        }
      },
      "number_of_shards": 5,
      "number_of_replicas": 2
    },
    "mappings": {
      "properties": {
        "id": {"type": "long"},
        "content": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "content_vector": {"type": "dense_vector","similarity": "cosine","index": true,"dims": 768,"element_type": "float","index_options": {"type": "hnsw","m": 16,"ef_construction": 128}},
        "content_answer": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "title": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "param": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "type": {"type": "keyword"},
        "questionId": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "createTime": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "updateTime": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "hitCount": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_smart"
        },
        "answerPattern": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "nearQuestionVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "questionEnclosureVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "questionRelationVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "rmsRoutingAnswerVos": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "label": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "question": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }
}'

PUT /_index_template/content_erp_nlp_help
{
  "index_patterns": ["content_erp*"],
  "priority": 100,
  "template": {
    "settings": {
      "analysis": {
        "analyzer": {
          "my_ik_analyzer": {
            "type": "ik_smart"
          }
        }
      },
      "number_of_shards": 5,
      "number_of_replicas": 2
    },
    "mappings": {
      "properties": {
        "id": {"type": "long"},
        "content": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "content_vector": {"type": "dense_vector","similarity": "cosine","index": true,"dims": 768,"element_type": "float","index_options": {"type": "hnsw","m": 16,"ef_construction": 128}},
        "content_answer": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "title": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "param": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "type": {"type": "keyword"},
        "questionId": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "createTime": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "updateTime": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "hitCount": {
          "type": "text",
          "analyzer": "ik_max_word",
          "search_analyzer": "ik_smart"
        },
        "answerPattern": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "nearQuestionVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "questionEnclosureVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "questionRelationVOList": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "rmsRoutingAnswerVos": {"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},
        "label": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "question": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }
}

查看模板信息

GET _index_template/content_erp_nlp_help

curl -X GET "http://localhost:9200/_index_template/content_erp_nlp_help"

删除模板的命令

DELETE _index_template/logs_index_template

curl -X DELETE "http://localhost:9200/_index_template/content_erp_nlp_help" -H "Content-Type: application/json"

查询线程信息

curl -X GET "localhost:9200/_cluster/settings?include_defaults=true&pretty"

创建索引

curl -X PUT "http://localhost:9200/content_erp_nlp_help_202503181600"

查看索引

curl -X GET "http://localhost:9200/_cat/indices?v"

创建别名

curl -X POST "http://localhost:9200/_aliases" -H "Content-Type: application/json" -d'
{
  "actions": [
    { "add": { "index": "content_erp_nlp_help_202503181600", "alias": "content_erp_nlp_help_alia" } }
  ]
}'

查看别名

curl -X GET "http://localhost:9200/_alias/content_erp_nlp_help_alia"

安装kibana

docker pull docker.elastic.co/kibana/kibana:8.14.1

mkdir kibana

cd /data/dockerinstall/kibana

mkdir data config

chmod 777 data/

上传kibana.yml 文件

server.host: "0.0.0.0"
# 连接es集群配置多个地址，单机一个地址
elasticsearch.hosts: ["http://172.16.108.43:9200"]
# 设置kibana中文
i18n.locale: "zh-CN"

启动
docker network create es-net

docker run -d --name kibana --network es-net -p 5601:5601 -v /data/dockerinstall/kibana/data:/usr/share/kibana/data -v /data/dockerinstall/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml docker.elastic.co/kibana/kibana:8.14.1

--------------------------------------------------------------------------------------------------------------------------------

安装过程中有问题时，需要删除ES实例再重启启动
先停止所有节点再删除，然后再重新启动所有节点，相当于重新安装

docker ps
docker stop 2df7dcd367e6
docker ps -a

docker rm 2df7dcd367e6

重新启动:

docker run -d --privileged=true --name es --restart=unless-stopped --network es-net -p 9200:9200 -p 9300:9300  -v /home/dockerinstall/es/data:/usr/share/elasticsearch/data  -v /home/dockerinstall/es/plugins:/usr/share/elasticsearch/plugins  -v /home/dockerinstall/es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml docker.elastic.co/elasticsearch/elasticsearch:8.14.1

查看基点状态只有两个点

curl -X GET "http://localhost:9200/_cat/nodes?v"

重新启动节点

有告警。

需要删除无用的索引

curl -X DELETE "http://localhost:9200/content_erp_nlp_help_202503181600"

本段无用请忽略，最后确认是服务器的互联互通问题

--------------------------------------------------------------------------------------------------------------------------------

ES 新建别名

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "content_erp_nlp_help_202503191415",
        "alias": "content_erp_nlp_help_alia"
      }
    }
  ]
}

获取别名

GET /_alias/content_erp_nlp_help_alia

删除别名中的索引

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "alias": "content_erp_nlp_help_alia",
        "index": "content_erp_nlp_help_202501091116"
      }
    }
  ]
}