python 分析 nginx 日志 ip 并存入sqlite 数据库

最新推荐文章于 2021-04-26 12:58:58 发布

最新推荐文章于 2021-04-26 12:58:58 发布 · 230 阅读

文章标签：

#nginx #Python #SQLite #C #C++

本文介绍了一个使用Python和SQLite处理特定目录下.log文件的方法。通过遍历文件夹中的所有日志文件，提取符合特定条件的数据，并将这些数据存入SQLite数据库中进行统计分析。

什么也不说，直接上代码

import sqlite3 import os import string import fileinput path_log_nginx = r"E:\xxx-logs\20100904" conn = sqlite3.connect('log.db') c = conn.cursor() #c.execute('''drop table log''') c.execute('''create table log(ip text)''') for file in os.listdir(path_log_nginx): if not file.endswith(".log"): print "WARN:%s is not a log file" %(file) continue print "INFO:process file %s" %(file) for line in fileinput.input(os.path.join(path_log_nginx, file)): if line.startswith("192.168."): continue; c.execute("insert into log (ip) values ('%s')" %(line[0:line.find(" - - [")])) conn.commit() #c.execute("select ip, count(*) c from log group by ip having c > 1000 order by c desc") # We can also close the cursor if we are done with it c.close() # print "done..."

完工后

sqlite3.exe log.db

>select ip, count(*) c from log group by ip having c > 1000 order by c desc;