Linux - 日志切割

背景

一般情况下，我们习惯于将各种应用程序（Web端程序、应用服务、数据库等）软件部署在 Linux操作系统上，但众多软件部署后运行时会产生对应的日志记录，以便于出现故障后能够及时排查问题原因。

随着日志文件的不断积累，占用的硬盘空间越来越大，达到GB级别后查看内容太耗时，要追踪错误等非常不方便，这对于运维、管理、故障排查等来说非常不方便。因此，需要对日志进行定期切割和清理。

日志切割介绍

日志切割：当应用程序或操作系统的日志文件满足设定的触发条件，对其进行切割。切割后的日志会在原有日志的基础上多出一个新的日志文件，且后续产生的日志也会被写入到新的日志文件中，直到下一次满足设定的触发条件时。

linux日志会定期进行滚动增加，需要在线对正在进行回滚的日志进行指定大小的切割（动态）

如果这个日志是静态的，可以使用split工具进行切割。

切割方式

1. 按时间切割：在进行切割日志时，以时间为标准，日志出现的时间满足设定的时间阈值时，则进行日志切割。例如：/var/log/messages 日志即按每7天切割一次的规则进行日志切分。

2. 按日志大小切割：在进行切割日志时，以日志大小为参考标准，日志的大小满足设定的大小时进行日志切割。一般应用程序的日志多使用容量进行切割。

Logrotate 介绍

Logrotate是一个Linux工具，用于管理系统中产生的日志文件。它可以自动将旧日志文件进行日志切割、压缩存档、删除和重命名等操作，以便释放磁盘空间。

优点：

1. logrotate 是Linux操作系统上自带的一款开源的日志切割软件（不用安装）

2. logrotate 自身已经集成进操作系统的定时任务中（不用配置定时任务）

3. logrotate 自身支持日志压缩

安装

[root@elk ~]# rpm -qa logrotate
logrotate-3.8.6-17.el7.x86_64

# 如果没有 logrotate
yum -y install logrotate

使用

1. 查看帮助文档

[root@elk ~]# logrotate --help
用法: logrotate [OPTION...] <configfile>
  -d, --debug               Don't do anything, just test (implies -v)
  -f, --force               Force file rotation
  -m, --mail=command        Command to send mail (instead of `/bin/mail')
  -s, --state=statefile     Path of state file
  -v, --verbose             Display messages during rotation
  -l, --log=STRING          Log file
  --version                 Display version information

Help options:
  -?, --help                Show this help message
  --usage                   Display brief usage message

选项

-d 	debug模式，测试配置文件是否有错误

-f 	强制转储文件

-m 	压缩日志后，发送日志到指定邮箱

-s 	执行状态指定输出

-v 输出日志分割执行的详细信息

-l 将执行日志存入指定文件

2. 查看版本

[root@elk /etc]# logrotate --version
logrotate 3.8.6

3. 配置文件

[root@elk /etc]# ls |grep logrotate
#主配置文件
logrotate.conf
#存放所有日志转存规则文件，logrotate执行日志转存任务时会读取该目录下所有文件
logrotate.d


#Logrotate是基于crond服务运行的，其脚本是/etc/cron.daily/logrotate，日志轮转是根据此计划任务自动完成的。此脚本文件不用修改，crond服务会自动调用logrotate.conf主配置文件。默认每天执行一次。
/etc/cron.daily/logrotate

参数解释

[root@elk /etc]# vim logrotate.conf

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# use date as a suffix of the rotated file
dateext

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
        minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.

weekly ：表示每周执行一次日志回滚

rotate：表示日志切割后，历史文件离现在最近的最多保存多少份

create ：指定新创建的文件权限、所属主和属组

dateext ：表示使用日期为后缀的回滚文件

/var/log/wtmp { # 指定日志文件的名字和路径
    missingok   # 如果文件丢失，不报错
    monthly     # 每月轮换一次
    create 0664 root utmp     # 指定 btmp 日志文件的权限、属主和属组
        minsize 1M         # 文件超过1M进行回滚
    rotate 1      # 日志切分后历史文件最多保存1份，不含当前使用的日志
}

compress: 在轮循任务完成后，已轮循的归档将使用gzip进行压缩。
delaycompress: 指示logrotate不要将最近的归档压缩，压缩将在下一次轮循周期进行。
notifempty: 如果日志文件为空，不进行轮询。
create 0644 root root: 以指定的文件权限创建全新的日志文件，同时logrotate也会重命名原始日志文件。
postrotate/endscript: postrotate/endscript里的命令最后执行。在这种情况下，rsyslogd 进程将立即再次读取其配置并继续运行。

4. 手动切割日志

logrotate -fv /etc/logrotate.conf

5. 查看日志切割记录

[root@elk ~]# cat /var/lib/logrotate/logrotate.status 
logrotate state -- version 2
"/var/log/yum.log" 2023-8-26-16:0:0
"/var/log/boot.log" 2023-9-12-12:36:1
"/var/log/chrony/*.log" 2023-8-26-16:0:0
"/var/log/wtmp" 2023-8-26-16:0:0
"/var/log/spooler" 2023-9-11-14:50:1
"/var/log/btmp" 2023-9-1-18:45:1
"/var/log/maillog" 2023-9-11-14:50:1
"/var/log/wpa_supplicant.log" 2023-8-26-16:0:0
"/var/log/secure" 2023-9-11-14:50:1
"/var/log/messages" 2023-9-11-14:50:1
"/var/log/cron" 2023-9-11-14:50:1

案例

对SSHD服务进行日志切割

1. 编辑一个sshd的配置文件

[root@elk /etc/logrotate.d]# ls
bootlog  chrony  sshd  syslog  wpa_supplicant  yum
[root@elk /etc/logrotate.d]# cat sshd
/var/log/sshd.log {
	missingok
	weekly
	create 0600 root root
	minsize 1M
	rotate 3
}

2. 重新启动rsyslog服务

systemctl start rsyslog

3. 轮询

[root@elk /etc/logrotate.d]# logrotate -d /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  weekly (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log does not need rotating (log has been already rotated)[root@elk /etc/logrotate.d]# 

强制轮询

         -v 显示指令执行过程

         -f 强制执行

[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/ssh.log forced from command line (2 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/ssh.log
  log /var/log/ssh.log does not exist -- skipping

[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
No logs found. Rotation not needed.

解决

[root@elk /var/log]# touch sshd.log

再次强制轮询

[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
old log /var/log/sshd.log.3 does not exist
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
old log /var/log/sshd.log.2 does not exist
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
old log /var/log/sshd.log.1 does not exist
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
log /var/log/sshd.log.4 doesn't exist -- won't try to dispose of it
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0

多次进行强制轮循，/var/log/sshd*文件只有3个

[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
old log /var/log/sshd.log.3 does not exist
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
old log /var/log/sshd.log.2 does not exist
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
old log /var/log/sshd.log.1 does not exist
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
log /var/log/sshd.log.4 doesn't exist -- won't try to dispose of it
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0
[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
old log /var/log/sshd.log.3 does not exist
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
old log /var/log/sshd.log.2 does not exist
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
log /var/log/sshd.log.4 doesn't exist -- won't try to dispose of it
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0
[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
old log /var/log/sshd.log.3 does not exist
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
log /var/log/sshd.log.4 doesn't exist -- won't try to dispose of it
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0
[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0
removing old log /var/log/sshd.log.4
[root@elk /etc/logrotate.d]# logrotate -vf /etc/logrotate.d/sshd
reading config file /etc/logrotate.d/sshd
Allocating hash table for state file, size 15360 B

Handling 1 logs

rotating pattern: /var/log/sshd.log  forced from command line (3 rotations)
empty log files are rotated, only log files >= 1048576 bytes are rotated, old logs are removed
considering log /var/log/sshd.log
  log needs rotating
rotating log /var/log/sshd.log, log->rotateCount is 3
dateext suffix '-20230914'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
renaming /var/log/sshd.log.3 to /var/log/sshd.log.4 (rotatecount 3, logstart 1, i 3), 
renaming /var/log/sshd.log.2 to /var/log/sshd.log.3 (rotatecount 3, logstart 1, i 2), 
renaming /var/log/sshd.log.1 to /var/log/sshd.log.2 (rotatecount 3, logstart 1, i 1), 
renaming /var/log/sshd.log.0 to /var/log/sshd.log.1 (rotatecount 3, logstart 1, i 0), 
old log /var/log/sshd.log.0 does not exist
renaming /var/log/sshd.log to /var/log/sshd.log.1
creating new /var/log/sshd.log mode = 0600 uid = 0 gid = 0
removing old log /var/log/sshd.log.4
[root@elk /etc/logrotate.d]# 

查看效果

[root@elk /var/log]# ls sshd*
sshd.log  sshd.log.1  sshd.log.2  sshd.log.3