The best way to deal with large files in Python style

原创于 2015-04-07 15:00:50 发布 · 525 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#python #文件

Python 专栏收录该内容

2 篇文章

订阅专栏

本文介绍了两种高效处理大文件的方法：使用with语句配合for循环逐行读取文件，以及利用生成器yield按块读取文件。这两种方法都能有效管理内存，避免因文件过大而导致的问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

<span style="font-family:Arial Black;font-size:12px;color:#000099;">with open(...) as f:
    for line in f:
        <do something with line></span>

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

Click here for origin.

Another alternative: by making use of yield.

<span style="font-family:Arial Black;font-size:12px;">def read_file(fpath): 
    BLOCK_SIZE = 1024 
    with open(fpath, 'rb') as f: 
        while True: 
            block = f.read(BLOCK_SIZE) 
            if block: 
                yield block 
            else: 
                return</span>

Click here for origin.