【爬虫】python 多线程知识

最新推荐文章于 2025-08-16 17:02:07 发布

转载最新推荐文章于 2025-08-16 17:02:07 发布 · 62 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/puyangsky/p/5338092.html

文章标签：

#python #爬虫

本文通过两个示例展示了Python中的多线程应用及并发编程技巧，包括线程类的实现、线程间的同步与通信，以及如何利用条件变量管理共享资源。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

第一段代码：

 1 __author__ = 'Administrator'
 2 
 3 import threading
 4 import time
 5 
 6 index = 0
 7 
 8 class MyThread(threading.Thread):
 9     def __init__(self, threadname, counter):
10         super(MyThread, self).__init__()
11         self.threadname = threadname
12         self.counter = counter
13 
14     def run(self):
15         global index
16         print "starting " + self.threadname
17         while self.counter:
18             time.sleep(1)
19             index += 1
20             print "threadname: %s\tindex:%d\ttime: %s\n"%(self.threadname, index, time.ctime())
21             self.counter -= 1
22 
23 def main():
24     print '=======================BEGIN==========================='
25     thread1 = MyThread('thread1', 10)
26     thread2 = MyThread('thread2', 5)
27     thread1.start()
28     thread2.start()
29 
30     thread1.join()
31     thread2.join()
32 
33     print '========================END============================'
34 
35 if __name__ == '__main__':
36     main()

第二段代码：

 1 # -*- coding: gbk -*-
 2 __author__ = 'Administrator'
 3 
 4 import threading
 5 import random, time, Queue
 6 
 7 
 8 # 缓冲队列大小为5
 9 MAX_SIZE = 5
10 # 模拟队列
11 SHARE_Q = []
12 # 条件变量
13 CONDITION = threading.Condition()
14 
15 
16 class Producer(threading.Thread):
17     def run(self):
18         products = range(5)
19         global SHARE_Q
20         while True:
21             # 获得锁
22             CONDITION.acquire()
23             # 队列如果满了，则等待资源释放
24             if len(SHARE_Q) == 5:
25                 print "Queue is full..."
26                 CONDITION.wait()
27                 print "Consumer have consumed something"
28             product = random.choice(products)
29             SHARE_Q.append(product)
30             print "Producer : ", product
31             CONDITION.notify()
32             CONDITION.release()
33             time.sleep(3)
34 
35 
36 class Consumer(threading.Thread):
37     def run(self):
38         global SHARE_Q
39         while True:
40             CONDITION.acquire()
41             # 队列为空，消费者不能进行消费
42             if not SHARE_Q:
43                 print "Queue is Empty..."
44                 CONDITION.wait()
45                 print "Producer have produced something"
46             product = SHARE_Q.pop(0)
47             print "Consumer: ", product
48             CONDITION.notify()
49             CONDITION.release()
50             time.sleep(2)
51 
52 
53 def main():
54     producer = Producer()
55     consumer = Consumer()
56     producer.start()
57     consumer.start()
58 
59 
60 if __name__ == '__main__':
61     main()