Centos 搭建 ELK 简明教程，收集 Nginx 日志显示

最新推荐文章于 2024-05-08 09:19:41 发布

原创最新推荐文章于 2024-05-08 09:19:41 发布 · 206 阅读

0 ·

CC 4.0 BY-SA版权

Elasticsearch 专栏收录该内容

4 篇文章

订阅专栏

本文详细介绍了如何在CentOS 7.5环境下部署Elasticsearch 7.3、Kibana 7.3和Logstash 7.3，涵盖了用户权限配置、软件下载与安装、配置修改及启动命令，确保日志数据的高效收集与分析。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

环境：

Centos: 7.5
Elasticsearch: 7.3
Kibana: 7.3
Logstash: 7.3

1. 准备

Linux 下安装 Elasticsearch 要使用非 root 账号，得新建一个。

adduser esuser //新建用户 
passwd esuser123 //给用户设置密码

给用户加 root 权限

修改 /etc/sudoers 文件，找到下面一行，把前面的注释（#）去掉

## Allows people in group wheel to run all commands
%wheel    ALL=(ALL)    ALL

然后修改用户，使其属于root组（wheel），命令如下：

usermod -g root esuser

2. 安装 elasticsearch-7.3.0

Linux 下安装方式，虚拟机里面 curl 下载慢的话可以用迅雷下载再上传文件。

curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.0-linux-x86_64.tar.gz
tar -xzvf elasticsearch-7.3.0-linux-x86_64.tar.gz
cd elasticsearch-7.3.0
./bin/elasticsearch  //启动ES

ES启动后，输入 curl http://127.0.0.1:9200 命令，正常的话会返回以下内容：

{
  "name" : "QtI5dUu",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "DMXhqzzjTGqEtDlkaMOzlA",
  "version" : {
    "number" : "7.3.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "00d8bc1",
    "build_date" : "2018-06-06T16:48:02.249996Z",
    "build_snapshot" : false,
    "lucene_version" : "7.3.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

3. 安装 Kibana

Kibana 是 elasticsearch 的可视化工作平台。

Linux 下安装 Kibana

curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-7.3.0-linux-x86_64.tar.gz
tar xzvf kibana-7.3.0-linux-x86_64.tar.gz
cd kibana-7.3.0-linux-x86_64/
./bin/kibana

启动前修改配置文件 /kibana/config/kibana.yml，将 server.host: "localhost" 改为下面这样，否则无法远程访问。

# Kibana is served by a back end server. This setting specifies the port to use.
#server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

好了，现在可以通过浏览器访问了 http://127.0.0.1:5601/

4. 安装 Logstash

$ wget https://artifacts.elastic.co/downloads/logstash/logstash-7.3.0.tar.gz
$ tar -zxvf logstash-7.3.0.tar.gz
$ cd /usr/local/logstash-7.3.0/bin
$ vim stdin.conf #编写配置文件
input {
    #这里可以同时监控多个文件
    file {
        path => ["/data/nginx/logs/error.log"]
        start_position => "beginning"
        type => "error"
    }
    file {
        path => ["/data/nginx/logs/access.log"]
        start_position => "beginning"
        type => "access"
    }
}
filter {
    #每种文件需要配置自己的grok插件语法来搜集需要的数据
    if [type] == "access"{
        grok {
            match => {
                #这里的须发需要自定义配置
                "message" => "^%{IPV4:remote_addr} \[%{HTTPDATE:timestamp}\] \"%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}\" %{INT:status} %{INT:body_bytes_sent} \"%{NOTSPACE:http_referer}\" %{NUMBER:request_time} \"%{IPV4:upstream_addr}:%{POSINT:upstream_port}\" %{NUMBER:upstream_response_time} \"%{DATA:http_user_agent}\" \"%{NOTSPACE:http_x_forwarded_for}\""
            }
        }
        #配置GeoIP的数据库解析ip
        geoip {
            source => "remote_addr"
        }   
    }
}
output {
        #数据输出到elasticsearch
        elasticsearch {
            hosts => ["127.0.0.1:9200"]
            index => "logstash-nginx-%{type}-%{+YYYY-MM}"
        }
    
    #调试
    stdout{codec => rubydebug}
}

$ ./logstash -f stdin.conf  #后台启动

NginxAccess日志格式：

log_format  main  '$remote_addr [$time_local] "$request" $status $body_bytes_sent "$http_referer" $request_time "$upstream_addr" $upstream_response_time "$http_user_agent" "$http_x_forwarded_for"';

浏览器访问下 nginx 的网站输出些日志，logstash 配置运行正常的话就会把日志写入到 ES 了，通过 Kibana 的LOGS查看：