《Elasticsearch核心技术与实践》学习笔记1、环境安装配置、数据准备-优快云博客

本文链接：https://blog.youkuaiyun.com/qililong88/article/details/109581968

本文介绍如何使用Docker Compose部署Elasticsearch、Kibana和Cerebro，并通过Logstash向Elasticsearch插入数据。提供了详细的docker-compose.yaml配置及Logstash配置样例。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在 Docker 容器中运行 Elasticsearch，Kibana（可视化UI维护数据）和 Cerebro（可视化UI维护服务），通过docker将几个服务同事启动：

docker-compose.yaml的内容：

version: '2.2'
services:
  cerebro:
    image: lmenezes/cerebro:0.8.3
    container_name: cerebro
    ports:
      - "9000:9000"
    command:
      - -Dhosts.0.host=http://elasticsearch:9200
    networks:
      - es7net
  kibana:
    image: docker.elastic.co/kibana/kibana:7.1.0
    container_name: kibana7
    environment:
      - I18N_LOCALE=zh-CN
      - XPACK_GRAPH_ENABLED=true
      - TIMELION_ENABLED=true
      - XPACK_MONITORING_COLLECTION_ENABLED="true"
    ports:
      - "5601:5601"
    networks:
      - es7net
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
    container_name: es7_01
    environment:
      - cluster.name=geektime
      - node.name=es7_01
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.seed_hosts=es7_01,es7_02
      - cluster.initial_master_nodes=es7_01,es7_02
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es7data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - es7net
  elasticsearch2:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
    container_name: es7_02
    environment:
      - cluster.name=geektime
      - node.name=es7_02
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.seed_hosts=es7_01,es7_02
      - cluster.initial_master_nodes=es7_01,es7_02
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - es7data2:/usr/share/elasticsearch/data
    networks:
      - es7net


volumes:
  es7data1:
    driver: local
  es7data2:
    driver: local

networks:
  es7net:
    driver: bridge

2、下载logstash(用于向ES插入数据，注意版本号要和ES相同):

下载地址

https://www.elastic.co/cn/downloads/past-releases#logstash

安装方法：https://blog.youkuaiyun.com/u013938578/article/details/104091791

并在解压后的logstash/bin目录下传入以下两文件logstash.conf以及

movies.csv（这是测试用例数据文件，不用上边的mysql数据，自己导入部分数据就可以使用了）

movies.csv下载地址：https://grouplens.org/datasets/movielens/

logstash.conf配置如下，注意用utf-8，防止编码错误

input {
  file {
    path => "/Users/qililong/movielens/logstash-7.1.0/bin/movies.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }

}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

运行命令： sudo ./logstash -f logstash.conf

可以看到插入的数据