ElasticSearch学习笔记之二：Filebeat日志收集

1. nginx日志收集

1.1 原生日志

• E：Elasticsearch
• F：Filebeat
• K：Kibana

[root@host3 ~]# vim 04-nginx-to-es.yml 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/access.log*
  tags: ["host3","nginx","access"]		# 打上标签，便于后期输出

- type: log
  paths:
    - /var/log/nginx/error.log*
  tags: ["host3","nginx","error"]
  fields:
    hostname: host3.test.com
    service: http
  fields_under_root: true       # 将fields字段提到最高级别

output.elasticsearch:
  hosts: ["http://192.168.19.101:9200","http://192.168.19.102:9200","http://192.168.19.103:9200"]
  indices:
    - index: "host3-nginx-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "access"
    - index: "host3-nginx-error-%{+yyyy.MM.dd}"
      when.contains:
        tags: "error"

# 关闭索引的生命周期，若开启则上面的index配置会被忽略
setup.ilm.enabled: false
# 设置索引模板的名称
setup.template.name: "host3-nginx"
# 设置索引模板的匹配模式
setup.template.pattern: "host3-nginx-*"
# 覆盖已有的索引模板
setup.template.overwrite: false
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 1

1.2 JSON格式日志

从nginx端输出的日志就已经分割好了，filebeat只需要接收并提取