logstash采集Java日志文本文件配合grok收集到elasticsearch简单示例

最新推荐文章于 2023-04-23 16:11:56 发布

Castle125

最新推荐文章于 2023-04-23 16:11:56 发布

阅读量886

点赞数

CC 4.0 BY-SA版权

分类专栏：运维 java 文章标签： log4j2 java

本文链接：https://blog.youkuaiyun.com/oCastle/article/details/108939983

本文提供了一个logstash配置示例，用于采集Java日志文本文件，通过grok解析后将数据送入Elasticsearch。内容包括logstash配置、日志文件格式以及在kibana中查看结果的方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

这里是引用

logstash采集Java日志文本文件配合grok收集到elasticsearch简单示例

logstash 配置
日志文件示例
kibana结果展示：
- 参考文章：

logstash 配置

input {
  file {
    path => "C:/Users/Administrator/Desktop/info*.log"
    type => "log-info"
    start_position => "beginning"
    # 多行文本换行处理 （日志文件以日期时间格式开头，所以按日期处理换行）
    codec => multiline {
    pattern => "^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}\s"
    negate => true
    what => "previous"
  }}
}
filter {
  grok {
  # 使用正则将日志拆分成多列，方便查询
  # 原日志文件使用log4j生成，格式为 ： %d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %class{36} %L %M - %msg%xEx%n
    match => { "message" => "(?<datetime>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{3}) %{NOTSPACE:level} (?<className> (?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*) %{NUMBER:line} %{NOTSPACE:method} - %{GREEDYDATA:msg}" }
	overwrite => ["message"]
  }
}
output {
	elasticsearch {
    hosts => "localhost:9200"
    index => "log4j-info-%{+YYYY.MM.dd}"
	codec => "json"
  }
}

日志文件示例

2020-08-10 14:47:56.204 INFO  org.springframework.boot.StartupInfoLogger 50 logStarting - Starting xxxxx on pc-111with PID 14520 (started by Administrator in E:\worker\XXXX\XXX\XXXX)
2020-08-10 14:47:56.214 INFO  org.springframework.boot.SpringApplication 679 logStartupProfileInfo - The following profiles are active: dev
2020-08-10 14:47:57.723 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 244 multipleStoresDetected - Multiple Spring Data modules found, entering strict repository configuration mode!
2020-08-10 14:47:57.733 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 126 registerRepositoriesIn - Bootstrapping Spring Data repositories in DEFAULT mode.
2020-08-10 14:47:57.803 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 182 registerRepositoriesIn - Finished Spring Data repository scanning in 50ms. Found 0 repository interfaces.
2020-08-10 14:47:57.813 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 244 multipleStoresDetected - Multiple Spring Data modules found, entering strict repository configuration mode!
2020-08-10 14:47:57.813 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 126 registerRepositoriesIn - Bootstrapping Spring Data repositories in DEFAULT mode.
2020-08-10 14:47:57.853 INFO  org.springframework.data.repository.config.RepositoryConfigurationDelegate 182 registerRepositoriesIn - Finished Spring Data repository scanning in 30ms. Found 0 repository interfaces.
2020-08-10 14:47:58.602 INFO  org.springframework.context.support.PostProcessorRegistrationDelegate$BeanPostProcessorChecker 330 postProcessAfterInitialization - Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$da77a69e] is not eligible for getting processed by all BeanPo