python 加载大文件，节省内存方法

1. 使用pandas处理csv文件

分块读取

chunksize = 1000
for chunk in pd.read_csv(csvpath, chunksize=chunksize):
    # 处理每个 chunk
    print(chunk)

2. python 自带csv包

按行读取

with open(csvpath, 'r', encoding='utf-8') as f:
    reader = csv.reader(f)
    
    # 按行读取文件
    for row in reader:
        print(row)