python--正则表达式过滤数据

最新推荐文章于 2024-09-12 03:55:52 发布

原创最新推荐文章于 2024-09-12 03:55:52 发布 · 1.4k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#pyton #正则

数据处理同时被 2 个专栏收录

13 篇文章

订阅专栏

python

11 篇文章

订阅专栏

该博客介绍了如何使用Python的正则表达式模块`re`解析文本文件，从日志中提取特定模式的数据。通过匹配模式，将数据转换为JSON格式，并根据不同的数据类型分别存储到`config_data`, `run_data`, 和 `send_msg`列表中。最终，将这些数据写入CSV文件进行保存。" 115523949,9238976,iLSMOA-2020：一种迭代问题重构的大规模多目标优化算法,"['优化算法', '多目标优化', '遗传算法', '进化计算', '机器学习']

import re
import csv
import json

CONFIG_HEADER = ["ChangeSrc", "timestamp", "power", "bright", "ct", "device_id"]
RUN_HEADER = ["ChangeSrc", "timestamp", "ManualAD", "PIR", "BrightAD", "device_id"]
SEND_HEADER = ["bright", "ct", "power", "L1", "bright_alpha", "bright_beta", "ct_alpha", "ct_beta", "changeSrc"]

def read_data(filename):
config_data = [CONFIG_HEADER]
run_data = [RUN_HEADER]
send_msg = [SEND_HEADER]
for line in open(filename):
match = re.match("(\[.*\]) (.*) message - (\d*): (\{.*\})", line.strip())
mode = match.group(2)
if mode == "recv":
message = json.loads(match.group(4))
if message["topic"] == "config":
config_data.append([message[i] for i in CONFIG_HEADER])
else:
run_data.append([message[i] for i in RUN_HEADER])
else:
message = json.loads(match.group(4))["outputs"][0]["fields"]
message = dict([(i["oKey"], i["oValue"]) for i in message])
send_msg.append([message[i] for i in SEND_HEADER])
return config_data, run_data, send_msg

def write_data(data, filename):
with open(filename, "wb") as f:
writer = csv.writer(f)
writer.writerows(data)

if __name__ == "__main__":
filename = "C:/Users/Administrator/Desktop/test002.txt"
config_data, run_data, send_msg = read_data(filename)
if len(config_data) > 1:
write_data(config_data, "C:/Users/Administrator/Desktop/config_data.csv")
if len(run_data) > 1:
write_data(run_data, "C:/Users/Administrator/Desktop/run_data.csv")
if len(send_msg) > 1:
write_data(send_msg, "C:/Users/Administrator/Desktop/send_msg.csv")