use another thread loading file for other threads parse will not speed up application

最新推荐文章于 2024-12-01 23:53:54 发布

最新推荐文章于 2024-12-01 23:53:54 发布 · 100 阅读

本文探讨了多线程在文件加载与解析过程中的实际效能提升问题。指出硬盘到内存的数据加载并非CPU密集型操作，因此专门用于加载数据的线程对于提升整体性能的作用有限。进一步讨论了当硬盘读取速度低于CPU处理速度时，单一线程负责加载即可，且通过数学模型展示了在特定条件下多CPU使用的最佳数量。

Multithreads load and parse file will notspeedup file handling. because load data from harddisk to memory is notthe CPU's business, so the only function the loading thread is bufferthe data for other threads, and the truth is harddisk already has itsown physical buffer used for speed up file loading, so for the big gdsloading case, use a unique thread loads file from harddisk will speedup nothing.

for another case, one thread load and several threads handle the data.

if HD data handling speed < CPU data handing speed, the CPU will alway stay hungry for data, so one thread is enough,

so now we suppose the hardisk handing data speed is 2 and CPU handling data speed is 1.

HD CPU

SPEED M/s 2 1

threads number 1 2 3

100% *1 100%*2 66%*3

This is the ideal stuation, and use 2CPUs aready reach the peak, if so how to speed up our application anddo we need a thread prepare data for other threads? yes, we can preparethe exact data for others help us to reach the ideal stuation, however,the speedup will not be significate and the job will be very verycomplex.