处理log文件

最新推荐文章于 2023-06-05 04:12:37 发布

JackLiu16

最新推荐文章于 2023-06-05 04:12:37 发布

阅读量636

点赞数

CC 4.0 BY-SA版权

分类专栏： python

本文链接：https://blog.youkuaiyun.com/JackLiu16/article/details/79201145

python 专栏收录该内容

208 篇文章

订阅专栏

本文介绍了一种从网络接口配置文件中提取IP地址与其对应接口名称的方法，并通过正则表达式匹配实现了自动化处理。同时，展示了如何统计日志文件中各IP地址对应的URL请求及其状态码出现的频次。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

#coding=utf-8
import re
import os
fileway = "D:\liujiaqi\Desktop\\test.txt"
match = {}
m1 = re.compile("flags")
m2 = re.compile("inet")
m3 = re.compile(":")
storefline = []
st = []
stto = []
with open(fileway) as f:
    for line in f:
        if m2.findall(line) or m1.findall(line):
            linee = line.split('\n')
            storefline.append(linee[0])
for li in storefline:
    if m3.findall(li):
        lie = li.split(':')
        st.append(lie[0])
    else:
        lie = li.strip().split(' ')
        stto.append(lie[1])
for i in range(len(st)):
    match.setdefault(st[i],stto[i])
print match
print st
print stto

{'log': '127.0.34.1', 'lo': '127.0.0.1', 'loh': '127.23.0.1', 'loo': '127.0.0.2', 'lol': '127.0.0.9', 'eth45': '172.31.28.7', 'lor': '127.0.0.134', 'lot': '127.234.0.1', 'eth8': '172.31.220.7', 'eth5': '172.31.228.234', 'eth3': '172.31.228.12', 'eth2': '172.234.228.7', 'eth1': '172.31.228.8', 'eth0': '172.31.228.7'}
['eth0', 'lo', 'eth1', 'loo', 'eth3', 'lol', 'eth5', 'lor', 'eth8', 'loh', 'eth45', 'lot', 'eth2', 'log']
['172.31.228.7', '127.0.0.1', '172.31.228.8', '127.0.0.2', '172.31.228.12', '127.0.0.9', '172.31.228.234', '127.0.0.134', '172.31.220.7', '127.23.0.1', '172.31.28.7', '127.234.0.1', '172.234.228.7', '127.0.34.1']
[Finished in 0.7s]

log文件：要求输入 ip得网口名称，输入网口名称得ip

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 172.31.228.7 netmask 255.255.240.0 broadcast 172.31.239.255
        ether 00:16:3e:01:f1:a9 txqueuelen 1000 (Ethernet)
        RX packets 3884253 bytes 1563509537 (1.4 GiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 3419061 bytes 1265800177 (1.1 GiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.1 netmask 255.0.0.0
        loop txqueuelen 1 (Local Loopback)
        RX packets 13816 bytes 7505494 (7.1 MiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 13816 bytes 7505494 (7.1 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 172.31.228.8 netmask 255.255.240.0 broadcast 172.31.239.255
        ether 00:16:3e:01:f1:a9 txqueuelen 1000 (Ethernet)
        RX packets 3884253 bytes 1563509537 (1.4 GiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 3419061 bytes 1265800177 (1.1 GiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

loo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.2 netmask 255.0.0.0
        loop txqueuelen 1 (Local Loopback)
        RX packets 13816 bytes 7505494 (7.1 MiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 13816 bytes 7505494 (7.1 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 172.31.228.12 netmask 255.255.240.0 broadcast 172.31.239.255
        ether 00:16:3e:01:f1:a9 txqueuelen 1000 (Ethernet)
        RX packets 3884253 bytes 1563509537 (1.4 GiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 3419061 bytes 1265800177 (1.1 GiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lol: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
        inet 127.0.0.9 netmask 255.0.0.0
        loop txqueuelen 1 (Local Loopback)
        RX packets 13816 bytes 7505494 (7.1 MiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 13816 bytes 7505494 (7.1 MiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
        inet 172.31.228.234 netmask 255.255.240.0 broadcast 172.31.239.255
        ether 00:16:3e:01:f1:a9 txqueuelen 1000 (Ethernet)
        RX packets 3884253 bytes 1563509537 (1.4 GiB)
        RX errors 0 dropped 0 overruns 0 frame 0
        TX packets 3419061 bytes 1265800177 (1.1 GiB)
        TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

补充优美代码：

# coding=utf-8
f = open('www_access_20140823.log')
res = {}
for l in f:
    arr = l.split(' ')
    # 获取ip url 和status
    ip = arr[0]
    url = arr[6]
    status = arr[8]
    # ip url 和status当key，每次统计+1
    res[(ip,url,status)] = res.get((ip,url,status),0)+1
# 生成一个临时的list
res_list = [(k[0],k[1],k[2],v) for k,v in res.items()]
# 按照统计数量排序，打印前10
for k in sorted(res_list,key=lambda x:x[3],reverse=True)[:10]:
    print k

('222.86.153.12', '/images/cursor_minify.cur', '404', 60)
('222.86.153.12', '/images/cursor_zoom.cur', '404', 32)
('58.253.6.133', '/images/cursor_minify.cur', '404', 32)
('111.85.34.165', '/%3Ca%20href=', '404', 28)
('58.253.6.133', '/images/cursor_zoom.cur', '404', 27)
('218.29.111.117', '/images/cursor_zoom.cur', '404', 27)
('218.29.111.117', '/images/cursor_minify.cur', '404', 26)
('117.63.146.40', '/public/js/common.js?20110824', '200', 19)
('117.63.146.40', '/favicon.ico', '404', 18)
('117.63.146.40', '/public/js/weibo.js?20110824', '200', 16)

log文件：
61.159.140.123 - - [23/Aug/2014:00:01:42 +0800] "GET /favicon.ico HTTP/1.1" 404 \ "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36 LBBROWSER" "-"
61.159.140.123 - - [23/Aug/2014:00:01:42 +0800] "GET /favicon.ico HTTP/1.1" 404 \ "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36 LBBROWSER" "-"
61.159.140.123 - - [23/Aug/2014:00:01:42 +0800] "GET /favicon.ico HTTP/1.1" 404 \ "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36 LBBROWSER" "-"
61.159.140.123 - - [23/Aug/2014:00:01:42 +0800] "GET /favicon.ico HTTP/1.1" 404 \ "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36 LBBROWSER" "-"
66.249.64.5 - - [23/Aug/2014:00:02:16 +0800] "GET /data/uploads/2013/0519/09/small_51982ba18e012.jpg HTTP/1.1" 200 \ "-" "Googlebot-Image/1.0" "-"
66.249.64.10 - - [23/Aug/2014:00:02:54 +0800] "GET /data/uploads/2013/0319/08/middle_5147b116e93b4.jpg HTTP/1.1" 200 \ "-" "Googlebot-Image/1.0" "-"