报错:
报错的filebeat-xxx.yml文件:
filebeat.inputs:
################ Web logs ################
- type: log
enabled: true
tail_files: true
scan_frequency: 5s
backoff: 1s
max_backoff: 10s
paths:
- /data/log/nginx/xxx.xxx.com*.log
tags: ["web", "xxx"]
fields_under_root: true
fields:
from: "xxx.xxx.com"
log_topic: "xxx-xxx-web"
################ App logs ################
- type: log
enabled: true
paths:
- /data/service/xxx/xxx/nohup.out
tags: ["xxxx"]
fields_under_root: true
fields:
from: "xxx-xxx"
log_topic: "xxx-xxx-app"
multiline.pattern: '^\d{4}\-\d{2}\-\d{2}\s\d+\:\d+\:\d+\.\d{2,12}\s'
multiline.negate: true
multiline.match: after
output.kafka:
enabled: true
hosts: ["x.x.x.x:xxxx"]
topic: '%{[fields.log_topic]}'
required_acks: 1
compression: gzip
keep_alive: 10s
报错原因:
因为output.kafka配置中的topic 取不到变量值,从而导致出现报错。
解决参考:
1、当配置fields_under_root: true时,output.kafka中的topic配置为topic: '%{[log_topic]}'
......
fields_under_root: true
fields:
from: "xxx-xxx"
log_topic: "xxx-xxx-app"
output.kafka
......
topic: '%{[log_topic]}'
......
2、当配置未fields_under_root: true时,output.kafka中的topic配置为topic: '%{[fileds.log_topic]}'
......
fields:
from: "xxx-xxx"
log_topic: "xxx-xxx-app"
output.kafka
......
topic: '%{[fileds.log_topic]}'
......