安装elasticsearch

E ： elasticsearch 搜索引擎 存储
L : logstash 日志收集
K : kibana 展示

Nginx: 负载均衡

业务场景:
日志:
app应用
win应用 ERP OA CRM B/S C/S
服务器日志
Tomcat、NGINX、http
DB MySQL

Nginx–>access.log–>L index名称–> E --> K

https://www.elastic.co/

日志易
splunk

一、使用背景

​ 当生产环境有很多服务器、很多业务模块的日志需要每时每刻查看时

二、环境

系统：centos 7

JDK：1.8

Elasticsearch-6.0.0

Logstash-6.0.0

kibana-6.0.0

三 安装java1.8

yum install java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64
[root@elk ~]# java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)

配置limit相关参数

[root@elk ~]# vim /etc/security/limits.conf
添加以下内容

[外链图片转存失败(img-wdic6GAB-1566354141764)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558880177764.png)]

[root@elk ~]# vim /etc/sysctl.conf

[外链图片转存失败(img-n8Fi3nZt-1566354141765)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558880630164.png)]

创建运行ELK的用户

[root@elk ~]# useradd elkuser

四：yum安装elasticsearch

[root@localhost ~]# yum install elasticsearch-6.0.0.rpm

配置文件：

[root@localhost ~]# vim /etc/elasticsearch/elasticsearch.yml

path.data: /var/lib/elasticsearch 

path.logs: /var/log/elasticsearch

network.host: 192.168.146.12

http.port: 9200

启动：

systemctl start elasticsearch

[外链图片转存失败(img-4RQANWlz-1566354141766)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558880916149.png)]

五：安装logstash：

[root@elk ~]# yum install logstash-6.0.0.rpm

vim /etc/logstash/conf.d/logstash-nginx-access-log.conf

input {
    file {
        path => ["/opt/nginx/logs/host.access.log"]    ###监控nginx的日志文件
        type => "nginx_access"
        start_position => "beginning"
    }
}

filter {
  grok {
    match => {
      "message" => '%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:time}\] "%{WORD:request_action} %{DATA:request} HTTP/%{NUMBER:http_version}" %{NUMBER:response} %{NUMBER:bytes} "%{DATA:referrer}" "%{DATA:agent}"'
    }
  }

  date {
    match => [ "time", "dd/MMM/YYYY:HH:mm:ss Z" ]
    locale => en
  }}

output {
  elasticsearch {
        hosts => ["192.168.146.12:9200"]
        index => "logstash-nginx-access-log"
    }
       stdout { codec => rubydebug }
}

接下里安装nginx

1、下载nginx源码包：curl -O https://nginx.org/download/nginx-1.12.2.tar.gz

2、安装nginx所需依赖：

​ gzip模块需要 zlib 库

​ rewrite模块需要 pcre 库

​ ssl 功能需要openssl库

yum -y install gcc* pcre pcre-devel zlib zlib-devel openssl openssl-devel

[root@elk ~]# tar -zxvf nginx-1.12.2.tar.gz -C /opt/ ###解压路径

• [root@elk opt]# mkdir nginx
  [root@elk opt]# cd nginx-1.12.2/
  [root@elk nginx-1.12.2]# ./configure --prefix=/opt/nginx
  
  checking for OS
  
  - Linux 3.10.0-693.el7.x86_64 x86_64
    checking for C compiler ... found
  - using GNU C compiler
  - gcc version: 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC)
  

  编译：make

  编译后安装：make install

启动nginx

/opt/nginx/logs/access.log ###这个就是需要监控的日志

启动logstash

[root@elk conf.d]# /usr/share/logstash/bin/logstash -f logstash-nginx-access-log.conf 使用这个配置文件启动

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console ###########

[外链图片转存失败(img-dwd87ZYR-1566354141768)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558882648816.png)]

logstash已经监控到nginx的access.log日志

六：接下来就是安装kibana把日志通过web界面显示出来

[root@elk ~]# yum install kibana-6.0.0-x86_64.rpm

[root@elk ~]# vim /etc/kibana/kibana.yml

该配置文件

server.host: "192.168.146.11"

elasticsearch.url: "http://192.168.146.11:9200"

systemctl start kibana

默认端口5601

[外链图片转存失败(img-AtRM2vc0-1566354141769)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558883791668.png)]

启动成功

logstash-* 这是索引匹配

[外链图片转存失败(img-xzxH0gEp-1566354141769)(C:\Users\成新健\AppData\Roaming\Typora\typora-user-images\1558883861467.png)]

日志的格式通过柱状图显示出来

很炫酷