python: file_handling 解决工作中出现的文件处理需求

本文介绍了一个Python脚本,用于在指定目录中搜索含有特定关键字的文件,并将找到的文件重命名为新的格式。此外,还介绍了如何利用Python标准库中的`shutil`和`copy`模块来实现文件操作和深拷贝。
 1 #!/usr/bin/python
 2 ## A file handling script, including functions that could search file by key
 3 ## and rename file by appending key value to the file names.
 4 
 5 import os
 6 import shutil
 7 import copy
 8 
 9 def get_INV_list(key_file):
10     """ read key_file and get key values into list"""
11     INV_file = open(key_file, 'r')
12     key_list = []
13     
14     for num in INV_file.readlines():
15         key_list.append([num[:-1]])
16     
17     INV_file.close()
18     return key_list
19 
20 def write_result_list(key_list,out_file):
21     """write updated key_list with reference into a file"""
22     result = open(out_file,'w')
23 
24     for key in key_list:
25         result.write('########'.join(key)+'\n')
26     
27     result.close()
28 
29 def search(key_list,search_dir):
30     """search key value in all files in search directory,
31        return a list with key reflect file name"""
32     
33     ##my original workaround
34     #result_list = []
35     #for item in key_list:
36     #    result_list.append(item[:])
37     ##A simpler code
38     #result_list = [item[:] for item in key_list]
39     ## or 
40     #result_list = [list(item) for item in key_list]
41     #best solution is copy.deepcopy()
42     result_list = copy.deepcopy(key_list)
43     
44     for file in os.listdir(search_dir):
45         search_file = open(search_dir+'/'+file,'r')
46         search_str = search_file.read()
47         search_file.close()
48         for key in result_list:
49             if key[0] == search_str[86:92]:    ##only check INV#
50             #if key[0] in search_str:
51                 key.append(file)
52     
53     return result_list
54 
55 def rename(result_list,input_dir,output_dir):
56     """ rename files in input directory and move to output directory,
57         return a list of file names those not moved."""
58     for item in result_list:
59         if len(item) > 1:
60             for filename in item[1:]:
61                 # move files to another folder and rename them
62                 # name format could be modified if necessary
63                 shutil.move(input_dir+'/'+filename,
64                         output_dir+'/'+filename+'.NZ'+item[0])
65 
66     return os.listdir(input_dir)
67 
68 xlist = get_INV_list('INV_Num')
69 ylist = search(xlist,'InputFiles')
70 rest_list = rename(ylist,'InputFiles','OutputFiles')
71 write_result_list(ylist,'resultList')

总结:

1. 之前已经写过一个简单脚本,对文件做关键字搜索以提取文件名,一周之后类似的需求只是需要修改对应文件名,不懂得代码重用,结果又折腾了两个小时才算是写好,今天有空将两段代码模块化,重构整个代码。

2. 学习到shutil, copy 模块,尤其是copy.deepcopy, 这个写法是从Stack Overflow 上提问才得到的,这是第一次提问,感觉国外的programmer回复很快,也很专业,以后要多去溜达学习。

3. 拼凑出来的代码,可以解决实际工作中的问题,这很重要,时不时的review这段代码,肯定还有许多可以改进之处。

转载于:https://www.cnblogs.com/Alex-Python-Waiter/archive/2012/07/09/2583631.html

PS D:\MACTN-main> & C:/Users/11245/AppData/Local/Programs/Python/Python313/python.exe d:/MACTN-main/2.py d:\MACTN-main\2.py:45: SyntaxWarning: invalid escape sequence '\d' return [int(text) if text.isdigit() else text.lower() for text in re.split('(\d+)', s)] Using device: cpu 开始加载 eeg_2hz 特征数据... 正在处理文件: 1_20151124_noon_2.mat (1/23) 正在处理文件: 2_20151106_noon.mat (2/23) 正在处理文件: 3_20151024_noon.mat (3/23) 正在处理文件: 4_20151105_noon.mat (4/23) 正在处理文件: 4_20151107_noon.mat (5/23) 正在处理文件: 5_20141108_noon.mat (6/23) 正在处理文件: 5_20151012_night.mat (7/23) 正在处理文件: 6_20151121_noon.mat (8/23) 正在处理文件: 7_20151015_night.mat (9/23) 正在处理文件: 8_20151022_noon.mat (10/23) 正在处理文件: 9_20151017_night.mat (11/23) 正在处理文件: 10_20151125_noon.mat (12/23) 正在处理文件: 11_20151024_night.mat (13/23) 正在处理文件: 12_20150928_noon.mat (14/23) 正在处理文件: 13_20150929_noon.mat (15/23) 正在处理文件: 14_20151014_night.mat (16/23) 正在处理文件: 15_20151126_night.mat (17/23) 正在处理文件: 16_20151128_night.mat (18/23) 正在处理文件: 17_20150925_noon.mat (19/23) 正在处理文件: 18_20150926_noon.mat (20/23) 正在处理文件: 19_20151114_noon.mat (21/23) 正在处理文件: 20_20151129_night.mat (22/23) 正在处理文件: 21_20151016_noon.mat (23/23) 拼接后的特征数据形状: (17, 20355, 25) Traceback (most recent call last): File "C:\Users\11245\AppData\Local\Programs\Python\Python313\Lib\site-packages\scipy\io\matlab\_mio.py", line 39, in _open_file return open(file_like, mode), True ~~~~^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'D:/data/SEED-VIG/perclos_labels\\perclos_labels.mat' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "d:\MACTN-main\2.py", line 351, in <module> main() ~~~~^^ File "d:\MACTN-main\2.py", line 222, in main eeg_data, labels = load_data('eeg_2hz', config['temporal_channel']) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "d:\MACTN-main\2.py", line 93, in load_data labels = loadmat(label_path)['perclos_labels'].squeeze() ~~~~~~~^^^^^^^^^^^^ File "C:\Users\11245\AppData\Local\Programs\Python\Python313\Lib\site-packages\scipy\io\matlab\_mio.py", line 233, in loadmat with _open_file_context(file_name, appendmat) as f: ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\11245\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 141, in __enter__ return next(self.gen) File "C:\Users\11245\AppData\Local\Programs\Python\Python313\Lib\site-packages\scipy\io\matlab\_mio.py", line 17, in _open_file_context f, opened = _open_file(file_like, appendmat, mode) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\11245\AppData\Local\Programs\Python\Python313\Lib\site-packages\scipy\io\matlab\_mio.py", line 45, in _open_file return open(file_like, mode), True ~~~~^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'D:/data/SEED-VIG/perclos_labels\\perclos_labels.mat'还是在报错找那个不存在的mat文件如何修改
06-05
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值