Log4J日志解析

最新推荐文章于 2021-02-16 14:57:32 发布
最新推荐文章于 2021-02-16 14:57:32 发布 · 413 阅读 · 2
文章标签：

#log4j #thread #F# #活动 #Cache

在有些场景下，需要解析Log4J的日志，以为己用。比如，根据关键字查询日志，用以定位问题等。就查询日志这个问题而论，通常的做法是登陆线上机器，grep一把日志目录，得到相关信息。这种做法有两个不好或者说不便捷的问题：首先要登陆线上机器，二来就是若有多台服务器，必须人肉的挨个儿查找，比较费时费力。

下面给出的解决方法是，分析log4j的配置文件，得到所有的Appender，然后再根据Appender中的日志路径、ConversionPattern等信息，去解析相应的日志文件，继而得到我们想要的信息，如Level，Logger、Message等信息。

闲话上说，直接上代码啦。

 

public final class ConversionRule {

	private boolean followedByQuotedString;
	private int beginIndex;
	private int length;
	private int minWidth = -1;
	private int maxWidth = -1;
	private String placeholderName;
	private String modifier;
	private Map<String, Object> properties = new HashMap<String, Object>();

	public boolean isFollowedByQuotedString() {
		return followedByQuotedString;
	}

	public void setFollowedByQuotedString(boolean followedByQuotedString) {
		this.followedByQuotedString = followedByQuotedString;
	}

	public int getBeginIndex() {
		return beginIndex;
	}

	public void setBeginIndex(int beginIndex) {
		this.beginIndex = beginIndex;
	}

	public int getLength() {
		return length;
	}

	public void setLength(int length) {
		this.length = length;
	}

	public int getMinWidth() {
		return minWidth;
	}

	public void setMinWidth(int minWidth) {
		this.minWidth = minWidth;
	}

	public int getMaxWidth() {
		return maxWidth;
	}

	public void setMaxWidth(int maxWidth) {
		this.maxWidth = maxWidth;
	}

	public String getPlaceholderName() {
		return placeholderName;
	}

	public void setPlaceholderName(String placeholderName) {
		this.placeholderName = placeholderName;
	}

	public String getModifier() {
		return modifier;
	}

	public void setModifier(String modifier) {
		this.modifier = modifier;
	}

	public void putProperty(String key, Object value) {
		properties.put(key, value);
	}

	@SuppressWarnings("unchecked")
	public <T> T getProperty(String key, Class<T> clazz) {
		return (T) properties.get(key);
	}

	@Override
	public String toString() {
		return "ConversionRule [modifier=" + modifier + ", placeholderName="   + placeholderName + "]";  
	}
public class ConversionRuleParser {

	private static final Pattern EXTRACTION_PATTERN = Pattern.compile("%(-?(\\d+))?(\\.(\\d+))?([a-zA-Z])(\\{([^\\}]+)\\})?"); 
	public static final String PROP_DATEFORMAT = "dateFormat";

	protected Pattern getInternalPattern(String externalPattern) throws Exception {
		List<ConversionRule> rules = extractRules(externalPattern);
		return Pattern.compile(toRegexPattern(prepare(externalPattern), rules));
	}
	
	protected List<ConversionRule> extractRules(String externalPattern) throws Exception {
		externalPattern = prepare(externalPattern);
		Matcher m = EXTRACTION_PATTERN.matcher(externalPattern);
		List<ConversionRule> ret = new ArrayList<ConversionRule>();
		while (m.find()) {
			String minWidthModifier = m.group(2);
			String maxWidthModifier = m.group(4);
			String conversionName = m.group(5);
			String conversionModifier = m.group(7);
			int minWidth = -1;
			if ((minWidthModifier != null) && (minWidthModifier.length() > 0)) {
				minWidth = Integer.parseInt(minWidthModifier);
			}
			int maxWidth = -1;
			if ((maxWidthModifier != null) && (maxWidthModifier.length() > 0)) {
				maxWidth = Integer.parseInt(maxWidthModifier);
			}
			ConversionRule rule = new ConversionRule();
			rule.setBeginIndex(m.start());
			rule.setLength(m.end() - m.start());
			rule.setMaxWidth(maxWidth);
			rule.setMinWidth(minWidth);
			rule.setPlaceholderName(conversionName);
			rule.setModifier(conversionModifier);
			rewrite(rule);
			ret.add(rule);
		}
		return ret;
	}

	public String prepare(String externalPattern) throws Exception {
		if (!externalPattern.endsWith("%n")) { 
			return externalPattern;
		}
		// Pattern without %n
		externalPattern = externalPattern.substring(0, externalPattern.length() - 2);
		if (externalPattern.contains("%n")) { 
			throw new Exception("ConversionPattern不合法!");
		}
		return externalPattern;
	}
	
	private void rewrite(ConversionRule rule) throws Exception {
		if (rule.getPlaceholderName().equals("d")) {
			applyDefaults(rule);
			if (rule.getModifier().equals("ABSOLUTE")) {
				rule.setModifier("HH:mm:ss,SSS");
			} else if (rule.getModifier().equals("DATE")) {
				rule.setModifier("dd MMM yyyy HH:mm:ss,SSS");
			} else if (rule.getModifier().equals("ISO8601")) {
				rule.setModifier("yyyy-MM-dd HH:mm:ss,SSS");
			}
			try {
				// Cache date format
				rule.putProperty(PROP_DATEFORMAT, new SimpleDateFormat(rule.getModifier()));
			} catch (IllegalArgumentException e) {
				throw new Exception(e);
			}
		}
	}

	private void applyDefaults(ConversionRule rule) throws Exception {
		if (rule.getModifier() == null) {
			// ISO8601 is the default
			rule.setModifier("ISO8601");
		}
	}

	private String getRegexPatternForRule(ConversionRule rule) throws Exception {
		if (rule.getPlaceholderName().equals("d")) {
			// Pattern is dynamic
			return "(" + RegexUtils.getRegexForSimpleDateFormat(rule.getModifier()) + ")";
		} else if (rule.getPlaceholderName().equals("p")) {
			String lnHint = RegexUtils.getLengthHint(rule);
			if (lnHint.length() > 0) {
				return "([ A-Z]" + lnHint + ")";
			}
			// Default: Length is limited by the levels available
			return "([A-Z]{4,5})";
		} else if (rule.getPlaceholderName().equals("c")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("t")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("m")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("F")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("C")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("M")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		} else if (rule.getPlaceholderName().equals("L")) {
			return "([0-9]*" + RegexUtils.getLengthHint(rule) + ")";
		} else if (rule.getPlaceholderName().equals("x")) {
			return "(.*" + RegexUtils.getLengthHint(rule) + RegexUtils.getLazySuffix(rule) + ")";
		}  
		throw new Exception("无法找到对应的表达式描述!");
	}

	protected String toRegexPattern(String externalPattern, List<ConversionRule> rules) throws Exception {
		// Determine whether rules are followed by quoted string, allowing use of special Regex lazy modifiers
		int idx = 0;
		ConversionRule prevRule = null;
		for (ConversionRule rule : rules) {
			if ((rule.getBeginIndex() > idx) && (prevRule != null)) {
				// Previous rule is followed by a quoted string, allowing special regex flags
				prevRule.setFollowedByQuotedString(true);
			}
			idx = rule.getBeginIndex();
			idx += rule.getLength();
			prevRule = rule;
		}
		if ((externalPattern.length() > idx) && (prevRule != null)) {
			// Previous rule is followed by a quoted string, allowing special regex flags
			prevRule.setFollowedByQuotedString(true);
		}
		// Build the internal Regex pattern
		StringBuilder sb = new StringBuilder();
		idx = 0;
		for (ConversionRule rule : rules) {
			if (rule.getBeginIndex() > idx) {
				// Escape chars with special meaning
				sb.append(Pattern.quote(externalPattern.substring(idx, rule.getBeginIndex())));
			}
			idx = rule.getBeginIndex();
			String regex = this.getRegexPatternForRule(rule);
			sb.append(regex);
			idx += rule.getLength();
		}
		if (externalPattern.length() > idx) {
			// Append suffix
			sb.append(Pattern.quote(externalPattern.substring(idx)));
		}
		return sb.toString();
	}
}

 

 

 

public class ConversionPatternParser {

	private Map<String, Appender> appenderBag;
	public static String CONVERSION_JUST_4TEST = "%d [] %-5p %c{2} - %m%n";

	public ConversionPatternParser() {}

	public void parseConfiguration(String configFilePath) throws Exception {
		AppenderParser config = new AppenderParser();
		config.parse(configFilePath);
		setAppenderBag(config.getAppenderBag());
	}

	public String getConversionPattern(Appender appender) {
		Layout layout = appender.getLayout();
		if (layout instanceof PatternLayout) {
			PatternLayout patternLayout = (PatternLayout) layout;
			return patternLayout.getConversionPattern();
		}
		return null;
	}

	public void setAppenderBag(Map<String, Appender> appenderBag) {
		this.appenderBag = appenderBag;
	}

	public Map<String, Appender> getAppenderBag() {
		return appenderBag;
	}
}
/**
 * @author Philipp Nanz
 */
public class RegexUtils {

	private static transient Logger logger = LoggerFactory.getLogger(RegexUtils.class);

	/**
	 * Returns the Regex lazy suffix for the given rule.
	 * @param rule the conversion rule
	 * @return the Regex lazy suffix
	 */
	public static String getLazySuffix(ConversionRule rule) {
		if (rule.isFollowedByQuotedString()) {
			return "?";
		} else {
			return "";
		}
	}

	/**
	 * Returns the Regex length hint for the given rule.
	 * @param rule the conversion rule
	 * @return the Regex length hint
	 */
	public static String getLengthHint(ConversionRule rule) {
		if ((rule.getMaxWidth() > 0) && (rule.getMaxWidth() == rule.getMinWidth())) {
			// Exact length specified
			return "{" + rule.getMaxWidth() + "}";
		} else if (rule.getMaxWidth() > 0) {
			// Both min and max are specified
			return "{" + Math.max(0, rule.getMinWidth()) + "," + rule.getMaxWidth() + "}"; //$NON-NLS-3$
		} else if (rule.getMinWidth() > 0) {
			// Only min is specified
			return "{" + rule.getMinWidth() + ",}";
		}
		return "";
	}

	/**
	 * Converts a given <code>java.lang.SimpleDateFormat</code> pattern into 
	 * a regular expression
	 * @param format the pattern
	 * @return the translated pattern
	 * @throws Exception if an error occurred
	 */
	public static String getRegexForSimpleDateFormat(String format) throws Exception {
		RegexUtils utils = new RegexUtils();
		return utils.doGetRegexForSimpleDateFormat(format);
	}

	private String doGetRegexForSimpleDateFormat(String format) throws Exception {
		try {
			new SimpleDateFormat(format);
		} catch (Exception e) {
			// Pattern is invalid
			throw new Exception(e);
		}
		// Initialize
		ReplacementContext ctx = new ReplacementContext();
		ctx.setBits(new BitSet(format.length()));
		ctx.setBuffer(new StringBuffer(format));
		// Unquote
		unquote(ctx);
		// G - Era designator
		replace(ctx, "G+", "[ADBC]{2}");
		// y - Year
		replace(ctx, "[y]{3,}", "\\d{4}");
		replace(ctx, "[y]{2}", "\\d{2}");
		replace(ctx, "y", "\\d{4}");
		// M - Month in year
		replace(ctx, "[M]{3,}", "[a-zA-Z]*");
		replace(ctx, "[M]{2}", "\\d{2}");
		replace(ctx, "M", "\\d{1,2}");
		// w - Week in year
		replace(ctx, "w+", "\\d{1,2}");
		// W - Week in month
		replace(ctx, "W+", "\\d");
		// D - Day in year
		replace(ctx, "D+", "\\d{1,3}");
		// d - Day in month
		replace(ctx, "d+", "\\d{1,2}");
		// F - Day of week in month
		replace(ctx, "F+", "\\d");
		// E - Day in week
		replace(ctx, "E+", "[a-zA-Z]*");
		// a - Am/pm marker
		replace(ctx, "a+", "[AMPM]{2}");
		// H - Hour in day (0-23)
		replace(ctx, "H+", "\\d{1,2}");
		// k - Hour in day (1-24)
		replace(ctx, "k+", "\\d{1,2}");
		// K - Hour in am/pm (0-11)
		replace(ctx, "K+", "\\d{1,2}");
		// h - Hour in am/pm (1-12)
		replace(ctx, "h+", "\\d{1,2}");
		// m - Minute in hour
		replace(ctx, "m+", "\\d{1,2}");
		// s - Second in minute
		replace(ctx, "s+", "\\d{1,2}");
		// S - Millisecond
		replace(ctx, "S+", "\\d{1,3}");
		// z - Time zone
		replace(ctx, "z+", "[a-zA-Z-+:0-9]*");
		// Z - Time zone
		replace(ctx, "Z+", "[-+]\\d{4}");
		return ctx.getBuffer().toString();
	}

	private void unquote(ReplacementContext ctx) {
		Pattern p = Pattern.compile("'[^']+'");
		Matcher m = p.matcher(ctx.getBuffer().toString());
		while (m.find()) {
			logger.trace(ctx.toString());
			// Match is valid
			int offset = -2;
			// Copy all bits after the match
			for (int i = m.end(); i < ctx.getBuffer().length(); i++) {
				ctx.getBits().set(i + offset, ctx.getBits().get(i));
			}
			for (int i = m.start(); i < m.end() + offset; i++) {
				ctx.getBits().set(i);
			}
			ctx.getBuffer().replace(m.start(), m.start() + 1, "");
			ctx.getBuffer().replace(m.end() - 2, m.end() - 1, "");
			logger.trace(ctx.toString());
		}
		p = Pattern.compile("''");
		m = p.matcher(ctx.getBuffer().toString());
		while (m.find()) {
			logger.trace(ctx.toString());
			// Match is valid
			int offset = -1;
			// Copy all bits after the match
			for (int i = m.end(); i < ctx.getBuffer().length(); i++) {
				ctx.getBits().set(i + offset, ctx.getBits().get(i));
			}
			for (int i = m.start(); i < m.end() + offset; i++) {
				ctx.getBits().set(i);
			}
			ctx.getBuffer().replace(m.start(), m.start() + 1, "");
			logger.trace(ctx.toString());
		}
	}

	private void replace(ReplacementContext ctx, String regex, String replacement) {
		Pattern p = Pattern.compile(regex);
		Matcher m = p.matcher(ctx.getBuffer().toString());
		while (m.find()) {
			logger.trace(regex);
			logger.trace(ctx.toString());
			int idx = ctx.getBits().nextSetBit(m.start());
			if ((idx == -1) || (idx > m.end() - 1)) {
				// Match is valid
				int len = m.end() - m.start();
				int offset = replacement.length() - len;
				if (offset > 0) {
					// Copy all bits after the match, in reverse order
					for (int i = ctx.getBuffer().length() - 1; i > m.end(); i--) {
						ctx.getBits().set(i + offset, ctx.getBits().get(i));
					}
				} else if (offset < 0) {
					// Copy all bits after the match
					for (int i = m.end(); i < ctx.getBuffer().length(); i++) {
						ctx.getBits().set(i + offset, ctx.getBits().get(i));
					}
				}
				for (int i = m.start(); i < m.end() + offset; i++) {
					ctx.getBits().set(i);
				}
				ctx.getBuffer().replace(m.start(), m.end(), replacement);
				logger.trace(ctx.toString());
			}
		}
	}

	private class ReplacementContext {

		private BitSet bits;
		private StringBuffer buffer;

		/**
		 * @return the bits
		 */
		public BitSet getBits() {
			return bits;
		}

		/**
		 * @param bits the bits to set
		 */
		public void setBits(BitSet bits) {
			this.bits = bits;
		}

		/**
		 * @return the buffer
		 */
		public StringBuffer getBuffer() {
			return buffer;
		}

		/**
		 * @param buffer the buffer to set
		 */
		public void setBuffer(StringBuffer buffer) {
			this.buffer = buffer;
		}

		/*
		 * (non-Javadoc)
		 * @see java.lang.Object#toString()
		 */
		@Override
		public String toString() {
			StringBuffer sb = new StringBuffer();
			sb.append("ReplacementContext [bits=");
			for (int i = 0; i < buffer.length(); i++) {
				sb.append(bits.get(i) ? '1' : '0');
			}
			sb.append(", buffer=");
			sb.append(buffer);
			sb.append(']');
			return sb.toString();
		}
	}
}
public class LogEntry {
	
	private Map<String, Object> map = new HashMap<String, Object>();
	 
	public final <VT> void put(String key, VT value) {
		map.put(key, value);
	}
 
	@SuppressWarnings("unchecked")
	public final <VT> VT get(String key) {
		return (VT) map.get(key);
	}
 
	public final <VT> boolean contains(String key) {
		return map.containsKey(key);
	}
	
	public String toString() {
		return map.toString();
	}
}
 

 

 

public class LogFileParser {

	public static final String FIELD_LOGGER = "logger";
	public static final String FIELD_TIMESTAMP = "timestamp";
	public static final String FIELD_LEVEL = "level";
	public static final String FIELD_THREAD = "thread";
	public static final String FIELD_MESSAGE = "message";
	public static final String FIELD_CURRENT_LINE = "currLine";
	public static final String FIELD_IS_HIT = "isHit";
	public static final String FIELD_NDC = "ndc";
	public static final String FIELD_THROWABLE = "throwable";
	public static final String FIELD_LOC_FILENAME = "locFilename";
	public static final String FIELD_LOC_CLASS = "locClass";
	public static final String FIELD_LOC_METHOD = "locMethod";
	public static final String FIELD_LOC_LINE = "locLine";
	private static Logger logger = Logger.getLogger(LogFileParser.class);
	private ConversionRuleParser conversionRuleParser;

	public LogFileParser() {
		conversionRuleParser = new ConversionRuleParser();
	}

	public List<LogEntry> parse(String fileName, String conversionPattern, String content, Integer upperLogNum, Integer lowerLogNum) throws Exception {
		List<ConversionRule> extractRules = conversionRuleParser.extractRules(conversionPattern);
		FileInputStream fis = new FileInputStream(new File(fileName));
		LineIterator iter = IOUtils.lineIterator(fis, "GBK");
		try {
			List<LogEntry> logLines = iterateLogLines(conversionPattern, extractRules, iter, upperLogNum, lowerLogNum, content);
			return logLines;
		} finally {
			LineIterator.closeQuietly(iter);
		}
	}

	@SuppressWarnings("unchecked")
	private List<LogEntry> iterateLogLines(String conversionPattern, List<ConversionRule> extractRules, LineIterator iter, Integer upperLogNum, Integer lowerLogNum, String content) throws Exception {
		boolean flag = true;
		List<LogEntry> result = new ArrayList<LogEntry>();
		BoundedFifoBuffer upperLogEntries = null;
		if (upperLogNum != null && upperLogNum > 0)
		    upperLogEntries = new BoundedFifoBuffer(upperLogNum);
		BoundedFifoBuffer lowerLogEntries = null;
		if (lowerLogNum != null && lowerLogNum > 0)
		    lowerLogEntries = new BoundedFifoBuffer(lowerLogNum);
		LogEntry unfinishedEntry = null;
		LogEntry currentEntry = fetchARecord(iter, conversionPattern, extractRules, unfinishedEntry);
		while (currentEntry != null) {
			String msg = currentEntry.get(FIELD_MESSAGE);
			boolean isHit = msg.contains(content);
			if (flag) {
				if (isHit) {
					//命中
					flag = false;
					if (upperLogEntries != null) {
						result.addAll(upperLogEntries);
						upperLogEntries.clear();
					}
					currentEntry.put(FIELD_IS_HIT, true);
					result.add(currentEntry);
				} else {
					if (upperLogEntries != null) {
						if (upperLogEntries.isFull()) {
							upperLogEntries.remove();
						}
						upperLogEntries.add(currentEntry);
					}
				}
				currentEntry = fetchARecord(iter, conversionPattern, extractRules, unfinishedEntry);
				continue;
			} else {
				if (!isHit) {
					if (lowerLogNum != 0) {
						//未命中
						if (lowerLogEntries != null) {
							lowerLogEntries.add(currentEntry);
							if (lowerLogEntries.isFull()) {
								//转移Lower中的记录到LogList中
								flag = true;
								result.addAll(lowerLogEntries);
								lowerLogEntries.clear();
							}
						}
					} else {
						flag = true;
					}
				} else {
					if (lowerLogEntries != null) {
						result.addAll(lowerLogEntries);
						lowerLogEntries.clear();
					}
					currentEntry.put(FIELD_IS_HIT, true);
					result.add(currentEntry);
				}
				currentEntry = fetchARecord(iter, conversionPattern, extractRules, unfinishedEntry);
				continue;
			}
		}
		return result;
	}

	private long lineNo = 1;

	private LogEntry fetchARecord(LineIterator iter, String conversionPattern, List<ConversionRule> extractRules, LogEntry unfinishedEntry) throws Exception {
		LogEntry currentEntry = null;
		boolean found = true;
		if (unfinishedEntry == null) {
			found = false;
		}
		if (!iter.hasNext()) {
			return null;
		}
		while (iter.hasNext()) {
			// Error handling 
			String line = iter.nextLine();
			while (StringUtils.isBlank(line) && iter.hasNext()) {
				line = iter.nextLine();
			}
			Matcher m = conversionRuleParser.getInternalPattern(conversionPattern).matcher(line);
			if (m.find()) {
				//It's next entry, unfinished
				if (found) {
					currentEntry = unfinishedEntry;
					unfinishedEntry = new LogEntry();
					for (int i = 0; i < m.groupCount(); i++) {
						try {
							this.extractField(unfinishedEntry, extractRules.get(i), m.group(i + 1));
						} catch (Exception e) {
							// Mark for interruption 
							logger.warn(e);
						}
					}
					currentEntry.put(FIELD_CURRENT_LINE, lineNo++);
					return currentEntry;
				} else {
					unfinishedEntry = new LogEntry();
					found = true;
					for (int i = 0; i < m.groupCount(); i++) {
						try {
							this.extractField(unfinishedEntry, extractRules.get(i), m.group(i + 1));
						} catch (Exception e) {
							// Mark for interruption 
							logger.warn(e);
						}
					}
				}
			} else if (unfinishedEntry != null) {
				String msg = unfinishedEntry.get(FIELD_MESSAGE);
				msg += '\n' + line;
				unfinishedEntry.put(FIELD_MESSAGE, msg);
			}
		}
		if (unfinishedEntry != null) {
			currentEntry = unfinishedEntry;
		}
		if (currentEntry != null)
		    currentEntry.put(FIELD_CURRENT_LINE, lineNo++);
		return currentEntry;
	}

	private void extractField(LogEntry entry, ConversionRule rule, String val) throws Exception {
		if (rule.getPlaceholderName().equals("d")) {
			DateFormat df = rule.getProperty(ConversionRuleParser.PROP_DATEFORMAT, DateFormat.class);
			entry.put(FIELD_TIMESTAMP, df.parse(val.trim()));
		} else if (rule.getPlaceholderName().equals("p")) {
			Level lvl = Level.toLevel(val.trim());
			entry.put(FIELD_LEVEL, lvl);
		} else if (rule.getPlaceholderName().equals("c")) {
			entry.put(FIELD_LOGGER, val.trim());
		} else if (rule.getPlaceholderName().equals("t")) {
			entry.put(FIELD_THREAD, val.trim());
		} else if (rule.getPlaceholderName().equals("m")) {
			entry.put(FIELD_MESSAGE, val.trim());
		} else if (rule.getPlaceholderName().equals("F")) {
			entry.put(FIELD_LOC_FILENAME, val.trim());
		} else if (rule.getPlaceholderName().equals("C")) {
			entry.put(FIELD_LOC_CLASS, val.trim());
		} else if (rule.getPlaceholderName().equals("M")) {
			entry.put(FIELD_LOC_METHOD, val.trim());
		} else if (rule.getPlaceholderName().equals("L")) {
			entry.put(FIELD_LOC_LINE, val.trim());
		} else if (rule.getPlaceholderName().equals("x")) {
			entry.put(FIELD_NDC, val.trim());
		} else {
			throw new Exception("异常消息暂未设置");
		}
	}
}

LogFileParser的活动图：

活动图

 

public class AppenderParser extends DOMConfigurator {

	private Map<String, Appender> appenderBag = new HashMap<String, Appender>();
	private Document doc = null;
	private DocumentBuilderFactory dbf;
	private DocumentBuilder db = null;

	public AppenderParser() {}

	public void parse(String configFile) throws Exception {
		doc = getDocument(configFile);
		NodeList appenderList = doc.getElementsByTagName("appender");
		for (int t = 0; t < appenderList.getLength(); t++) {
			Node node = appenderList.item(t);
			NamedNodeMap map = node.getAttributes();
			Node attrNode = map.getNamedItem("name");
			if (getAppenderBag().get(attrNode.getNodeValue()) == null) {
				Appender appender = parseAppender((Element) node);
				getAppenderBag().put(attrNode.getNodeValue(), appender);
			}
		}
	}

	private Document getDocument(String configFile) throws ParserConfigurationException, SAXException, IOException {
		dbf = DocumentBuilderFactory.newInstance();
		db = dbf.newDocumentBuilder();
		return db.parse(new File(configFile));
	}
	
	public static void main(String[] args) throws Exception {
		String configFile = "D:\\log4j.xml";
	    AppenderParser config = new AppenderParser();
		config.parse(configFile);
		System.out.println(SerializeUtil.serialize(config.getAppenderBag()));
    }

	public void setAppenderBag(Map<String, Appender> appenderBag) {
	    this.appenderBag = appenderBag;
    }

	public Map<String, Appender> getAppenderBag() {
	    return appenderBag;
    }
}
@Component
public class LoggerSearch {

	private AppenderParser appenderParser;
	private LogFileParser logParser;
	private Map<String, Appender> appenders;
	private String configFilePath = "log4j.xml";
	List<LogEntry> logEntries;
	Map<String, List<LogEntry>> allLogEntries;

	@PostConstruct
	public void init() throws Exception {
		appenderParser = new AppenderParser();
		logParser = new LogFileParser();
		appenderParser.parse(getConfigFilePath());
		appenders = appenderParser.getAppenderBag();
		allLogEntries = new HashMap<String, List<LogEntry>>();
	}

	public Map<String, List<LogEntry>> searchAll(String content, Integer upperLogNum, Integer lowerLogNum) throws Exception {
		for (Appender appender : appenders.values()) {
			if (appender instanceof FileAppender) {
				FileAppender fileAppender = (FileAppender) appender;
				if (appender instanceof DailyRollingFileAppender) {
					Layout layout = fileAppender.getLayout();
					if (layout instanceof PatternLayout) {
						PatternLayout patternLayout = (PatternLayout) layout;
						String conversionPattern = patternLayout.getConversionPattern();
						String fileName = fileAppender.getFile();
						logEntries = logParser.parse(fileName, conversionPattern, content, upperLogNum, lowerLogNum);
						allLogEntries.put(new File(fileName).getName(), logEntries);
					}
				}
			}
		}
		return allLogEntries;
	}

	public void setConfigFilePath(String configFilePath) {
		this.configFilePath = configFilePath;
	}

	public String getConfigFilePath() {
		return LoggerSearch.class.getClassLoader().getResource(configFilePath).getFile();
	}
}

 

 

上述部分代码参考了LogSaw，其中仅实现了简单的功能，如果有需要可自行扩展。

另附上一个JMX查询接口：

 

@Component
@JmxClass
public class Log4jGrep {

	@Resource
	private LoggerSearch loggerSearch;

	@JmxMethod
	public String searchAll(String content, Integer upperLogNum, Integer lowerLogNum) {
		try {
			StringBuilder sb = new StringBuilder();
			Map<String, List<LogEntry>> logEntries = loggerSearch.searchAll(content, upperLogNum, lowerLogNum);
			String color = "#cccccc";
			for (String logName : logEntries.keySet()) {
				List<LogEntry> logs = logEntries.get(logName);
				if (logs != null && logs.size() > 0) {
					sb.append("<p style=\"\font-weight: bold;color: #FF0000;font-family: Geneva, Arial, Helvetica, sans-serif;font-size: x-large;\">" + logName + "</p>");
					sb.append("<table height=\"100\" border=\"1\">");
					sb.append("<tr>");
					sb.append("<td width=\"45\">行号</td>");
					sb.append("<td width=\"55\">等级</td>");
					sb.append("<td width=\"88\">类名</td>");
					sb.append("<td width=\"763\">信息</td>");
					sb.append("</tr>");
					for (LogEntry log : logs) {
						if (log.get(LogFileParser.FIELD_IS_HIT) != null) {
							color = "#cccccc";
						} else {
							color = "";
						}
						sb.append("<tr bgcolor=" + color + ">");
						sb.append("<td>" + log.get(LogFileParser.FIELD_CURRENT_LINE) + "</td>");
						sb.append("<td>" + log.get(LogFileParser.FIELD_LEVEL) + "</td>");
						sb.append("<td>" + log.get(LogFileParser.FIELD_LOGGER) + "</td>");
						String msg = log.get(LogFileParser.FIELD_MESSAGE);
						msg = msg.replaceAll(content, "<span style=\"color: #FFFFFF;background: #FF0000;\">" + content + "</span>");
						sb.append("<td style=\"word-wrap:break-word;\">" + msg + "</td>");
						sb.append("</tr>");
					}
					sb.append("<table/>");
				}
			}
			return sb.toString();
		} catch (Exception e) {
			return ExceptionUtils.getFullStackTrace(e);
		}
	}
}
搭建简易elk+log4j的日志分析系统
newayue的博客
03-17 1826
因为公司业务需要要进行日志分析，所以就需要搭建一个日志采集和分析系统。而elk集采集，存储，分析和查询于一身，并且elk系统本身也比较成熟。一：基础配置elk组件都是基于java的，所以jdk是必须要到，最好是安装1.8版本，因为了解到最新版本的elk对于java版本有要求。二：安装elasticsearch：1：下载下载网站：https://www.elastic.co/downloads；为避...
Log4j日志框架讲解（全面，详细）
最新发布
smilehjl
07-02 3033
Log4j是Apache下的一款开源的日志框架，通过在项目中使用 Log4J，我们可以控制日志信息输出到控制台、文件、甚至是数据库中。我们可以控制每一条日志的输出格式，通过定义日志的输出级别，可以 更灵活的控制日志的输出过程。方便项目的调试。log4j这个日志框架是现在那些流行的日志框架实现的始祖，现在流行的logback,log4j2这些都是基于或者参考log4j这个日志框架实现的，所以学习log4j可以让我们更好的理解和使用现在流行的主流日志框架。
log4j日志详细讲解
08-16
log4j日志。。。。。。。。。。。。。。。。。。。。。。。。。。。。。
chainsaw软件log4j日志查看软件
03-06
chainsaw软件-log4j日志查看软件
log4j日志采集工具
06-12
采用tcp方式采集分布在各个机器上的log4j产生的日志，详细的使用方法可以参见我的博客，地址https://blog.youkuaiyun.com/qq_33222871/article/details/80662899
Log4j 日志信息存储到数据库中
03-04
log4j 日志信息存储到数据库的相关配置与使用
Log4j日志记录框架配置及用法解析
08-18
Log4j日志记录框架配置及用法解析 Log4j是Apache的一个开源日志组件，广泛应用于Java项目中，主要用于记录系统日志，追溯问题， debug和优化系统性能。在实际开发中，Log4j的配置和使用非常重要，本文将详细介绍Log...
原生Apache Log4J API打造的Log4J日志解析器
2. **配置解析器**: 配置解析器以识别和解析Log4J日志文件。这可能包括定义输入源、格式化规则等。 3. **使用解析后的数据**: 编写代码处理解析后的数据，可以是存储、分析、可视化或警报等操作。 ### 知识点六：与...
log4j-parser:使用本机Apache Log4J API的Log4J解析器
05-28
"log4j-parser"是针对Log4j日志文件设计的一个解析器，它利用Apache Log4J的本地API来高效地处理和解析日志数据。 **1. Log4j基本概念** Log4j由三个主要组件构成：Logger（日志器）、Appender（输出端）和Layout...
Tomcat使用Log4j输出catalina.out日志
01-10
日志格式和项目中用log4j打出来的不一致，不利于解析。 从tomcat官网（https://tomcat.apache.org/tomcat-7.0-doc/logging.html）上找了下，修改一些配置、替换扩展包即可使用log4j输出catalian.out。 在$CATALINA_...
Log4j的配置方法
xhf0813的专栏
05-09 373
这里用的是org.apache.log4j.Logger log4j.rootLogger=error, stdout   log4j.appender.appenderName=org.apache.log4j.AsyncAppender   log4j.appender.stdout=org.apache.log4j.ConsoleAppe
Log4j日志详解
程序人生
06-08 385
一.Log4j简介Log4j有三个主要的组件：Loggers(记录器)，Appenders(输出源)和Layouts(布局)。这里可简单理解为日志类别，日志要输出的地方和日志以何种形式输出。综合使用这三个组件可以轻松的记录信息的类型和级别，并可以在运行时控制日志输出的样式和位置。综合使用这三个组件可以轻松的记录信息的类型和级别，并可以在运行时控制日志输出的样式和位置。1.LoggersLogger...
Log4j的日志级别分析（转）
weixin_33941350的博客
11-07 114
说明：Log4j的日志是有级别的，从低到高顺序为：ALL &lt; DEBUG &lt; INFO &lt; WARN &lt; ERROR &lt; FATAL &lt; OFF，当定义了日志级别为WARN后，那么其比它高的级别（ERROR &lt; FATAL）会打印出来。 基本使用方法： Log4j由三个重要的组件构成：日志信息的优先级，日志信息的输出目的地，日志信息的输出格式。日志信息...
Log4j采集日志分析(模拟)
someInNeed的博客
08-11 563
test下新建directory然后idea的右上角项目结构使其变成Test，新建LogGenerator import org.apache.log4j.Logger; public class LoggerGenerator { private static Logger logger = Logger.getLogger(LoggerGenerator.class.getNa...
Java开发日志分析工具Log4j使用详解
cheney521的专栏
01-23 9108
常用的日志控件有commons-logging、Log4j以及JDK1.4自带的Logging。 日志简介： 早期，java开发用System.out.println(),输出到屏幕，但是这种输出不可控制，日志的可读性差，而且输出到屏幕需要消耗资源，大量的IO操作会使程序运行变慢。 JKD1.4自带的Logging，从JDK1.4起，JDK开始自带一套日志系统。JDK Logging最大的优
java 日志文件解析_java – 从日志文件解析Log4j布局
weixin_32568295的博客
02-16 339
以下是Chainsaw用于将patternlayouts转换为有用的代码的代码,仅使用log4j API.对于Chainsaw的最新开发人员快照,它用于直接从log4j xml或属性文件fileappender条目构建Chainsaw配置.使用PatternParser#parse填充的转换器和字段来执行您想要的操作. PatternParser类包含在log4j’adts’伴侣中.public ...
Log4j解析
Kobe_DHP的博客
01-05 487
一.Log4j原理 本文是在Linux操作系统基础上利用log4j产生日志文件来模拟日志服务器不断产生日志的行为，若读者想要在Eclipse上运行只需稍作修改，非常简单，在此，不作过多阐述。流程图如图1-1所示。                                                                                          ...
log4j日志基础详解
weixin_39456575的博客
12-23 275
工作中，需要的日志过滤不出来。没用的日志打印一大堆，着实是一件令人头疼的事情。 因为对日志基础都不了解，跟我们的开发小哥哥提了一个意见。 我：“胖白，咱们的日志能不能优化一下？每次都筛选不到我要的日志，而且咱们的日志磁盘都不够...” 胖白：“......噢，这个可以先关闭整体的DEBUG日志，然后通过相关的工程拿到对应的类，打开相关类DEBUG日志就好。要不再去了解一下log4j日志的一些基础知识吧？（小声的补充一句）” 为此，我整理了一些相关的日志基础，很快就能了解到打印日志、调整日志打印级别等
log4j日志详解
不列颠
11-16 722
Log4j使用指南 1         概述 本文档是针对Log4j日志工具的使用指南。包括：日志介绍、日志工具介绍、Log4j基本使用、Log4j的高级使用、Spring与log4j的集成等。并进行了举例说明。 本文档适合所有Java开发人员。 2         日志介绍 存储软件程序、服务或操作系统产生的消息记录的文件。 电脑里的日志是指日志数据可以是有价值的信息宝库，也
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值