Filebeat和Elasticsearch之间HTTPS连接

参考：https://www.elastic.co/guide/en/beats/filebeat/current/elasticsearch-output.html

https://www.elastic.co/guide/en/beats/filebeat/current/securing-communication-elasticsearch.html

https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-basic-setup-https.html

官方网站给我们提供了一个设置 Metricbeat 到 Elasticsearch 的 HTTPS 连接的例子，beats 到 Elasticsearch 的 HTTPS 连接都是一样的，所以下面的 Filebeat 到 Elasticsearch 的 HTTPS 连接是参考官网的。

还是在之前的 docker-compose.yml 文件的基础上进行修改，之前我们已经完成了 Kibana 到 Elasticsearch 之间的HTTPS连接，Kibana 使用 elasticsearch-ca.pem 文件来连接 Elasticsearch，Filebeat 同样也是使用该文件来连接 Elasticsearch。

1. 修改 Filebeat 的配置文件

#输入配置
filebeat.inputs:
    - type: log
    enabled: true
    paths:
    #需要收集的日志文件所在位置，可用通配符
    - /root/work/logs/*.log

output.elasticsearch:
    hosts: ["es01:9200"]
    protocol: "https"
    username: "elastic"
    password: "123123"
    ssl:
      certificate_authorities: ["elasticsearch-ca.pem"]
      verification_mode: "certificate"

2. 修改 docker-compose.yml 文件，将 elasticsearch-ca.pem 挂载到 Filebeat 的安装目录下

version: '2.2'
services: 
  es01:
    image: elasticsearch:7.14.1
    container_name: es01
    environment:
      - node.name=es01
      - discovery.seed_hosts=es02
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      #开启内存锁定检查
      - bootstrap.memory_lock=true
      #限制堆大小
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      #开启安全功能
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.client_authentication=required
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.keystore.path=http.p12
      - xpack.security.http.ssl.truststore.path=http.p12
      #注意这里，默认是full,会校验主机名，如果在生成证书的时候没有设置主机名，这里改成certificate
      - xpack.security.http.ssl.verification_mode=certificate
    volumes:
      - /root/work/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - /root/work/cert/https/elasticsearch/http.p12:/usr/share/elasticsearch/config/http.p12
    #内存锁定
    ulimits: 
      memlock:
        soft: -1
        hard: -1
  es02:
    image: elasticsearch:7.14.1
    container_name: es02
    environment:
      - node.name=es02
      - discovery.seed_hosts=es01
      - cluster.initial_master_nodes=es01,es02
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate 
      - xpack.security.transport.ssl.client_authentication=required
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.keystore.path=http.p12
      - xpack.security.http.ssl.truststore.path=http.p12
      #注意这里，默认是full,会校验主机名，如果在生成证书的时候没有设置主机名，这里改成certificate
      - xpack.security.http.ssl.verification_mode=certificate
    volumes:
      - /root/work/elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
      - /root/work/cert/https/elasticsearch/http.p12:/usr/share/elasticsearch/config/http.p12
    ulimits:
      memlock:
        soft: -1
        hard: -1
  kibana:
    image: kibana:7.14.1
    container_name: kibana
    environment:
      - SERVER_NAME=kibana.localhost
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - I18N_LOCALE=zh-CN
      - ELASTICSEARCH_USERNAME=elastic
      - ELASTICSEARCH_PASSWORD="123123"
      - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY="fhjskloppd678ehkdfdlliverpoolfcr"
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/elasticsearch-ca.pem
      #注意这里，如果是full,会校验主机名，如果在生成证书的时候没有设置主机名，这里改成certificate
      - ELASTICSEARCH_SSL_VERIFICATIONMODE=certificate
    volumes:
      - /root/work/cert/https/kibana/elasticsearch-ca.pem:/usr/share/kibana/config/elasticsearch-ca.pem
    ports:
      - 5601:5601
    depends_on:
      - es01
  filebeat:
    image: elastic/filebeat:7.14.1
    container_name: filebeat
    volumes:
      - /root/work/beats/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml
      - /root/work/logs/:/root/work/logs/
      - /root/work/cert/https/kibana/elasticsearch-ca.pem:/usr/share/filebeat/elasticsearch-ca.pem
    depends_on:
      - es01

3. 启动

docker-compose up -d

4. 给 Elasticsearch 集群设置用户名和密码，也是要和配置文件中设置的密码一致，其实可以写一个自动设置用户名密码的脚本，不然每个重启都要设置用户名和密码，太麻烦了。
5. 访问 Kibana

可以看到 filebeat 开头的索引就是从 filebeat 收集来的数据。