python爬虫(多线程&锁&信号量)

最新推荐文章于 2025-07-07 15:48:16 发布

原创最新推荐文章于 2025-07-07 15:48:16 发布 · 2.1k 阅读

1 ·

CC 4.0 BY-SA版权

python 专栏收录该内容

19 篇文章

订阅专栏

本文通过三个具体示例介绍Python中多线程的应用，包括无锁多线程、使用锁的多线程以及利用信号量控制并发数量的方法。每个示例都详细展示了如何启动线程并执行指定任务。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1、多线程（无锁）：threading.Thread(target = ***).start()

  1 import threading
  2 import time
  3 import random
  4 
  5 def worker_func():
  6         print('worker thread is started at %s' % threading.current_thread())
  7         random.seed()
  8         time.sleep(random.random())
  9         print('worker thread is finished at %s' % threading.current_thread())
 10 
 11 def simple_thread_demo():
 12         for i in range(10):
 13                 threading.Thread(target=worker_func).start()
 14 
 15 if __name__ == '__main__':
 16         simple_thread_demo()

2、多线程（锁 & 单线程）：<1> threading.Lock() <2> threading.Thread(target = ***,args = [***]).start()

  1 #coding=utf-8
  2 import threading
  3 import random
  4 import time
  5 
  6 def worker_func():
  7         print('worker thread is started at %s' % threading.current_thread())
  8         random.seed()
  9         time.sleep(random.random())
 10         print('worker thread is finished at %s' % threading.current_thread())
 11 
 12 def worker_func_lock(mylock):
 13         mylock.acquire()
 14         worker_func()
 15         mylock.release()
 16 
 17 gLock = threading.Lock()        #创建线程锁
 18 
 19 def thread_lock_demo():
 20         for i in range(10):
 21                 threading.Thread(target=worker_func_lock,args=[gLock]).start()  #开启线程
 22 
 23 if __name__ == '__main__':
 24         thread_lock_demo()

3、多线程（信号量 & 多线程）：<1> threading.Semaphore() <2> threading.Thread(target = ***,args = [***]).start()

  1 #coding=utf-8
  2 import threading
  3 import random
  4 import time
  5 
  6 def worker_func():
  7         print('worker thread is started at %s' % threading.current_thread())
  8         random.seed()
  9         time.sleep(random.random())
 10         print('worker thread is finished at %s' % threading.current_thread())
 11 
 12 def worker_func_lock(mylock):
 13         mylock.acquire()
 14         worker_func()
 15         mylock.release()
 16 
 17 gSem = threading.Semaphore(3)   #信号量为3
 18 
 19 def thread_lock_demo():
 20         for i in range(10):
 21                 threading.Thread(target=worker_func_lock,args=[gSem]).start()   #开启线程
 22 
 23 if __name__ == '__main__':
 24         thread_lock_demo()