docker部署ELK

启动elasticsearch

• 拉取elasticsearch镜像

docker pull elasticsearch:7.8.0

• 启动elasticsearch容器

docker run -d -p 9200:9200 -p 9300:9300 -e "http.cors.enabled=true" -e "http.cors.allow-origin=*" -e "discovery.type=single-node" --name elasticsearch elasticsearch:7.8.0

启动kibana

• 拉取kibana镜像

docker pull kibana:7.8.0

• 启动kibana容器

docker run --name kibana -d -p 5601:5601 --link elasticsearch:elasticsearch kibana:7.8.0

启动logstash

• 拉取logstash镜像

docker pull logstash:7.8.0

• 启动logstash容器

docker run -d -v /logs:/logs -p 5044:5044 --name logstash logstash:7.8.0

• 进入容器logstash

docker exec -it logstash bash

• 修改容器logstash配置文件

bash-4.2$ vi config/logstash.yml 
bash-4.2$ cat config/logstash.yml 
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://192.168.0.148:9200" ]

bash-4.2$ vi /usr/share/logstash/pipeline/logstash.conf
bash-4.2$ cat /usr/share/logstash/pipeline/logstash.conf
input{
	file{
		codec=>json
		path=>"/logs/*.json"
	}
}
filter{
	grok{
		match=>{
			"message"=>"%{DATA:timestamp}\|%{IP:serverIp}\|%{IP:clientIp}\|%{DATA:logSource}\|%{DATA:userId}\|%{DATA:reqUrl}\|%{DATA:reqUri}\|%{DATA:refer}\|%{DATA:device}\|%{DATA:textDuring}\|%{DATA:duringTime:int}\|\|"
		}
	}
}
output{
	elasticsearch{
		hosts=>"http://192.168.0.148:9200"
	}
}

• 验证容器logstash配置文件是否正确

bash-4.2$ logstash -f /usr/share/logstash/pipeline/logstash.conf -t
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2024-10-25T16:22:54,512][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2024-10-25T16:22:55,831][INFO ][org.reflections.Reflections] Reflections took 47 ms to scan 1 urls, producing 21 keys and 41 values 
Configuration OK
[2024-10-25T16:22:56,424][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

• 退出logstash容器

bash-4.2$ exit
exit
(base) [root@localhost logs]# 

• 重启logstash容器

(base) [root@localhost logs]# docker restart logstash
logstash
(base) [root@localhost logs]# 

• 往/logs/a.json写入数据

(base) [root@localhost logs]# echo '{"serverIp":"xxxxxxxxxxxxxxxxx"}' >> /logs/a.json 
(base) [root@localhost logs]# 

• 配置kibana页面

如下图所示表示配置完成

点击如下图所示

此时就能看到数据