Ubuntu中ELK安装和调试的一些要点

1. Query all records from Elasticsearch

curl -XGET "http://localhost:9200/logstash-*/_search?size=50&pretty"

http://stackoverflow.com/a/38874465/2177408

2. Run logstash

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

3. Linux 命令（netstat, ps, kill）

On Linux (Ubuntu derivatives at least)

killall node
is easier than this form.

ps | grep <something>
kill <somepid>
Neither will work if you have a orphaned child holding the port. Instead, do this:

netstat -punta | grep <port>
If the port is being held you'll see something like this:

tcp           0      0.0.0.0:<port>          0.0.0.*       LISTEN     <pid>/<parent>
Now kill by pid:

kill -9 <pid>

http://stackoverflow.com/a/22875192/2177408

4. Delete all data of an index in elasticsearch

curl -XDELETE localhost:9200/index/type/documentID

http://stackoverflow.com/a/22932471/2177408

5. Match IPV4 and IPV6 in COMBINEDAPACHELOGS

%{IPV6:ipv6}:%{IPV4:ipv4}

http://stackoverflow.com/a/40084695/2177408

6. Multiple matches in grok in logstash

input {
  stdin{}
}
filter {
  grok {
    break_on_match => false
    match => [ "message", "%{WORD:word1}" ]
    match => [ "message", "%{WORD:word2}" ]
    match => [ "message", "%{WORD:word3}" ]
  }
}
output {
  stdout { codec => rubydebug }
}

7. Regex for COMBINEDAPACHELOGS

grok {
  match => [ "message", "%{IP:client_ip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:apache_timestamp}\] \"%{WORD:method} /%{NOTSPACE:request_page} HTTP/%{NUMBER:http_version}\" %{NUMBER:server_response} " ]
}

http://stackoverflow.com/a/22380896/2177408

8. After updating logstash.conf

rm /var/lib/logstash/.sincedb*
curl -XDELETE localhost:9200/logstash-*
sudo /etc/init.d/logstash stop
sudo /etc/init.d/logstash start

9. Kibana 自动关闭问题的解决

kibana在aws上跑一段时间就关闭了，不知道为什么。

NODE_OPTIONS="--max-old-space-size=200" bin/kibana

https://github.com/elastic/kibana/issues/5170

https://github.com/elastic/kibana/issues/6153#issuecomment-189718168