spring-boot-logback-SizeAndTimeBasedRollingPolicy-totalSizeCap失效问题排查

前提说明

现状

1. logback版本：1.2.3
2. 使用spring-boot自带的logback作为日志工具：

       <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
         <fileNamePattern>${LOG_DIR}/%d{yyyyMMdd}/${LOG_LEVEL}/${LOG_FILE}.%i</fileNamePattern>
         <maxFileSize>${LOG_FILE_MAX_SIZE:-10MB}</maxFileSize>
         <maxHistory>${LOG_FILE_MAX_HISTORY:-7}</maxHistory>
         <totalSizeCap>${LOG_FILE_TOTAL_SIZE_CAP:-100G}</totalSizeCap>
         <cleanHistoryOnStart>false</cleanHistoryOnStart>
       </rollingPolicy>

   配置文件如上

3. 问题：压测机器时发现最大100G的限制没有正常生效。

问题排查

1. 本地限制totalSizeCap为30M。编写测试程序

   @Slf4j
   public class Starter implements ApplicationRunner {
   ......
   
     /**
      * Callback used to run the bean.
      *
      * @param args incoming application arguments
      * @throws Exception on error
      */
     @Override
     public void run(ApplicationArguments args) throws Exception {
       new Thread(()->{
         while (true){
           log.info("测试日志");
         }
       }).start();
     }
   }

    

2. 进行测试发现日志限制正常。
3. 推断是压测原因导致的日志判断异常。

logback源码

     定时清理启动流程：

1. 启动定时任务

   public class TimeBasedArchiveRemover extends ContextAwareBase implements ArchiveRemover {
   ......
     //启动定时任务
     public Future<?> cleanAsynchronously(Date now) {
           ArhiveRemoverRunnable runnable = new ArhiveRemoverRunnable(now);
           ExecutorService executorService = context.getScheduledExecutorService();
           Future<?> future = executorService.submit(runnable);
           return future;
       }
      public class ArhiveRemoverRunnable implements Runnable {
           Date now;
   
           ArhiveRemoverRunnable(Date now) {
               this.now = now;
           }
   
           @Override
           public void run() {
               //执行定时任务
               clean(now);
               if (totalSizeCap != UNBOUNDED_TOTAL_SIZE_CAP && totalSizeCap > 0) {
                   capTotalSize(now);
               }
           }
       }
       void capTotalSize(Date now) {
           long totalSize = 0;
           long totalRemoved = 0;
           for (int offset = 0; offset < maxHistory; offset++) {
               Date date = rc.getEndOfNextNthPeriod(now, -offset);
               //获取匹配的日志File数组
               File[] matchingFileArray = getFilesInPeriod(date);
               //给匹配的数组排序，最后修改的放前面
               descendingSortByLastModified(matchingFileArray);
               for (File f : matchingFileArray) {
                   long size = f.length();
                   if (totalSize + size > totalSizeCap) {
                       addInfo("Deleting [" + f + "]" + " of size " + new FileSize(size));
                       totalRemoved += size;
                       f.delete();
                   }
                   totalSize += size;
               }
           }
           addInfo("Removed  " + new FileSize(totalRemoved) + " of files");
       }
       protected File[] getFilesInPeriod(Date dateOfPeriodToClean) {
           File archive0 = new File(fileNamePattern.convertMultipleArguments(dateOfPeriodToClean, 0));
           File parentDir = getParentDir(archive0);
           //这里的表达式是info.log.(\d{1,3})
           String stemRegex = createStemRegex(dateOfPeriodToClean);
           File[] matchingFileArray = FileFilterUtil.filesInFolderMatchingStemRegex(parentDir, stemRegex);
           return matchingFileArray;
       }
       private String createStemRegex(final Date dateOfPeriodToClean) {
           String regex = fileNamePattern.toRegexForFixedDate(dateOfPeriodToClean);
           return FileFilterUtil.afterLastSlash(regex);
       }
   ......
   }

   public class FileNamePattern extends ContextAwareBase {
   ......
      public String toRegexForFixedDate(Date date) {
           StringBuilder buf = new StringBuilder();
           Converter<Object> p = headTokenConverter;
           while (p != null) {
               if (p instanceof LiteralConverter) {
                   buf.append(p.convert(null));
               } else if (p instanceof IntegerTokenConverter) {
                   buf.append("(\\d{1,3})");
               } else if (p instanceof DateTokenConverter) {
                   buf.append(p.convert(date));
               }
               p = p.getNext();
           }
           return buf.toString();
       }
   ......
   }

    

2. 很明显这里用于文件匹配的表达式错了，info.log.(\d{1,3}) 最多只能匹配999序号的数据。超过的就无法被匹配了

3. 那么要解决这个问题，办法也就很简单了

解决办法

     调高每个日志文件大小，保证序号不要超过999即可,这个值根据每日产生的日志量做调整，50MB不是绝对的。

     <maxFileSize>${LOG_FILE_MAX_SIZE:-50MB}</maxFileSize>