今天总结一下Python的Multi-Processing多线程编程。
创建多进程的方法
start new thread
使用thread.start_new_thread是一种最简单的创建多进程的方法。
#!/usr/bin/python
import thread
import time
# Define a function for the thread
def print_time(threadName, delay):
count = 0
while count < 5:
time.sleep(delay)
count += 1
print "%s: %s"%(threadName, time.ctime(time.time()))
# Create two threads as follows
try:
thread.start_new_thread(print_time, ("Thread-1", 2,))
thread.start_new_thread(print_time, ("Thread-2", 4,))
except:
print "Error: unable to start thread"
while 1:
pass
输出结果为:
Thread-1: Wed Sep 27 19:52:57 2017
Thread-2: Wed Sep 27 19:52:59 2017
Thread-1: Wed Sep 27 19:52:59 2017
Thread-1: Wed Sep 27 19:53:01 2017
Thread-1: Wed Sep 27 19:53:03 2017
Thread-2: Wed Sep 27 19:53:03 2017
Thread-1: Wed Sep 27 19:53:05 2017
Thread-2: Wed Sep 27 19:53:07 2017
Thread-2: Wed Sep 27 19:53:11 2017
Thread-2: Wed Sep 27 19:53:15 2017
如果没有程序末尾的while无限循环,thread1和thread2之外的主线程瞬间就会结束,程序就退出了,因此会什么也不打印。
threading module
#!/usr/bin/python
import threading
import time
exitFlag = 0
class myThread(threading.Thread):
def __init__(self, threadID, name, delay):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.delay = delay
def run(self):
print "Starting " + self.name
print_time(self.name, 5, self.delay)
print "Exiting " + self.name
def print_time(threadName, counter, delay):
while counter:
if exitFlag:
threadName.exit()
time.sleep(delay)
print "%s: %s" % (threadName, time.ctime(time.time()))
counter -= 1
# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)
# Start new Threads
thread1.start()
thread2.start()
print "Exiting Main Thread"
输出结果为:
Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Wed Sep 27 20:00:16 2017
Thread-2: Wed Sep 27 20:00:17 2017
Thread-1: Wed Sep 27 20:00:17 2017
Thread-1: Wed Sep 27 20:00:18 2017
Thread-2: Wed Sep 27 20:00:19 2017
Thread-1: Wed Sep 27 20:00:19 2017
Thread-1: Wed Sep 27 20:00:20 2017
Exiting Thread-1
Thread-2: Wed Sep 27 20:00:21 2017
Thread-2: Wed Sep 27 20:00:23 2017
Thread-2: Wed Sep 27 20:00:25 2017
Exiting Thread-2
也可以通过加入线程列表的形式,等待线程执行结束后主线程才退出:
#!/usr/bin/python
import threading
import time
exitFlag = 0
class myThread(threading.Thread):
def __init__(self, threadID, name, delay):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.delay = delay
def run(self):
print "Starting " + self.name
print_time(self.name, 5, self.delay)
print "Exiting " + self.name
def print_time(threadName, counter, delay):
while counter:
if exitFlag:
threadName.exit()
time.sleep(delay)
print "%s: %s" % (threadName, time.ctime(time.time()))
counter -= 1
# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)
# Start new Threads
thread1.start()
thread2.start()
# Add threads to thread list
threads = []
threads.append(thread1)
threads.append(thread2)
# Wait for all threads to complete
for t in threads:
t.join()
print "Exiting Main Thread"
输出结果为:
Starting Thread-1
Starting Thread-2
Thread-1: Wed Sep 27 20:04:21 2017
Thread-1: Wed Sep 27 20:04:22 2017
Thread-2: Wed Sep 27 20:04:22 2017
Thread-1: Wed Sep 27 20:04:23 2017
Thread-2: Wed Sep 27 20:04:24 2017
Thread-1: Wed Sep 27 20:04:24 2017
Thread-1: Wed Sep 27 20:04:25 2017
Exiting Thread-1
Thread-2: Wed Sep 27 20:04:26 2017
Thread-2: Wed Sep 27 20:04:28 2017
Thread-2: Wed Sep 27 20:04:30 2017
Exiting Thread-2
Exiting Main Thread
互斥锁同步线程
不加锁的问题
当不同的线程操作同一批数据的时候,就有可能产生意想不到的问题。比如说:
# encoding: UTF-8
import threading
import time
class MyThread(threading.Thread):
def run(self):
global num
time.sleep(1)
num = num+1
msg = self.name+' set num to '+str(num)
print msg
num = 0
threads = []
def test():
for i in range(1000):
t = MyThread()
t.start()
threads.append(t)
if __name__ == '__main__':
test()
for t in threads:
t.join()
print num
理论上说,一千个线程都让num增1,结果应该是1000,但是输出的结果不是1000:(而且每次都不一样)
............
Thread-998 set num to 905
Thread-988 set num to 906
Thread-994 set num to 904
Thread-992 set num to 901
Thread-990 set num to 907
Thread-1000 set num to 908
908
这是因为不同线程同时操作同样数据导致的线程不同步。此时需要通过加互斥锁实现线程的同步。
加锁的效果
# encoding: UTF-8
import threading
import time
class MyThread(threading.Thread):
def run(self):
global num
time.sleep(1)
threadLock.acquire()
num = num+1
threadLock.release()
msg = self.name+' set num to '+str(num)
print msg
num = 0
threadLock = threading.Lock()
threads = []
def test():
for i in range(1000):
t = MyThread()
t.start()
threads.append(t)
if __name__ == '__main__':
test()
for t in threads:
t.join()
print num
加锁相当于每个数据在每一时刻只能被一个线程获取到,而获取不到锁的线程就会处于暂时的阻塞状态。输出结果是这样的:
.........
Thread-994 set num to 994
Thread-995 set num to 995
Thread-993 set num to 993
Thread-996 set num to 996
Thread-999 set num to 997
Thread-1000 set num to 998
Thread-998 set num to 999
Thread-997 set num to 1000
1000
可以看出,并不是按照线程的创建时间获取数据的处理权限,但是结果却可以保证每个线程都能依次处理数据,从而保证数据被自增了1000次。