Python Multi-Processing多线程编程

今天总结一下Python的Multi-Processing多线程编程。

创建多进程的方法

start new thread

使用thread.start_new_thread是一种最简单的创建多进程的方法。

#!/usr/bin/python

import thread
import time

# Define a function for the thread
def print_time(threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s"%(threadName, time.ctime(time.time()))

# Create two threads as follows
try:
   thread.start_new_thread(print_time, ("Thread-1", 2,))
   thread.start_new_thread(print_time, ("Thread-2", 4,))
except:
   print "Error: unable to start thread"

while 1:
   pass

输出结果为：

Thread-1: Wed Sep 27 19:52:57 2017
Thread-2: Wed Sep 27 19:52:59 2017
Thread-1: Wed Sep 27 19:52:59 2017
Thread-1: Wed Sep 27 19:53:01 2017
Thread-1: Wed Sep 27 19:53:03 2017
Thread-2: Wed Sep 27 19:53:03 2017
Thread-1: Wed Sep 27 19:53:05 2017
Thread-2: Wed Sep 27 19:53:07 2017
Thread-2: Wed Sep 27 19:53:11 2017
Thread-2: Wed Sep 27 19:53:15 2017

如果没有程序末尾的while无限循环，thread1和thread2之外的主线程瞬间就会结束，程序就退出了，因此会什么也不打印。

threading module

#!/usr/bin/python

import threading
import time

exitFlag = 0

class myThread(threading.Thread):

   def __init__(self, threadID, name, delay):
      threading.Thread.__init__(self)
      self.threadID = threadID
      self.name = name
      self.delay = delay

   def run(self):
      print "Starting " + self.name
      print_time(self.name, 5, self.delay)
      print "Exiting " + self.name

def print_time(threadName, counter, delay):
   while counter:
      if exitFlag:
         threadName.exit()
      time.sleep(delay)
      print "%s: %s" % (threadName, time.ctime(time.time()))
      counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

print "Exiting Main Thread"

输出结果为：

Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Wed Sep 27 20:00:16 2017
Thread-2: Wed Sep 27 20:00:17 2017
Thread-1: Wed Sep 27 20:00:17 2017
Thread-1: Wed Sep 27 20:00:18 2017
Thread-2: Wed Sep 27 20:00:19 2017
Thread-1: Wed Sep 27 20:00:19 2017
Thread-1: Wed Sep 27 20:00:20 2017
Exiting Thread-1
Thread-2: Wed Sep 27 20:00:21 2017
Thread-2: Wed Sep 27 20:00:23 2017
Thread-2: Wed Sep 27 20:00:25 2017
Exiting Thread-2

也可以通过加入线程列表的形式，等待线程执行结束后主线程才退出：

#!/usr/bin/python

import threading
import time

exitFlag = 0

class myThread(threading.Thread):

   def __init__(self, threadID, name, delay):
      threading.Thread.__init__(self)
      self.threadID = threadID
      self.name = name
      self.delay = delay

   def run(self):
      print "Starting " + self.name
      print_time(self.name, 5, self.delay)
      print "Exiting " + self.name

def print_time(threadName, counter, delay):
   while counter:
      if exitFlag:
         threadName.exit()
      time.sleep(delay)
      print "%s: %s" % (threadName, time.ctime(time.time()))
      counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

# Add threads to thread list
threads = []
threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()

print "Exiting Main Thread"

输出结果为：

Starting Thread-1
Starting Thread-2
Thread-1: Wed Sep 27 20:04:21 2017
Thread-1: Wed Sep 27 20:04:22 2017
Thread-2: Wed Sep 27 20:04:22 2017
Thread-1: Wed Sep 27 20:04:23 2017
Thread-2: Wed Sep 27 20:04:24 2017
Thread-1: Wed Sep 27 20:04:24 2017
Thread-1: Wed Sep 27 20:04:25 2017
Exiting Thread-1
Thread-2: Wed Sep 27 20:04:26 2017
Thread-2: Wed Sep 27 20:04:28 2017
Thread-2: Wed Sep 27 20:04:30 2017
Exiting Thread-2
Exiting Main Thread

互斥锁同步线程

不加锁的问题

当不同的线程操作同一批数据的时候，就有可能产生意想不到的问题。比如说：

# encoding: UTF-8
import threading
import time

class MyThread(threading.Thread):
    def run(self):
        global num
        time.sleep(1)
        num = num+1
        msg = self.name+' set num to '+str(num)
        print msg
num = 0
threads = []
def test():
    for i in range(1000):
        t = MyThread()
        t.start()
    threads.append(t)
if __name__ == '__main__':
    test()
    for t in threads:
        t.join()
    print num

理论上说，一千个线程都让num增1,结果应该是1000,但是输出的结果不是1000:(而且每次都不一样)

............
Thread-998 set num to 905
Thread-988 set num to 906
Thread-994 set num to 904
Thread-992 set num to 901
Thread-990 set num to 907
Thread-1000 set num to 908
908

这是因为不同线程同时操作同样数据导致的线程不同步。此时需要通过加互斥锁实现线程的同步。

加锁的效果

# encoding: UTF-8
import threading
import time

class MyThread(threading.Thread):
    def run(self):
        global num
        time.sleep(1)
    threadLock.acquire() 
        num = num+1
    threadLock.release()
        msg = self.name+' set num to '+str(num)
        print msg
num = 0
threadLock = threading.Lock()
threads = []
def test():
    for i in range(1000):
        t = MyThread()
        t.start()
    threads.append(t)
if __name__ == '__main__':
    test()
    for t in threads:
        t.join()
    print num

加锁相当于每个数据在每一时刻只能被一个线程获取到，而获取不到锁的线程就会处于暂时的阻塞状态。输出结果是这样的：

.........
Thread-994 set num to 994
Thread-995 set num to 995
Thread-993 set num to 993
Thread-996 set num to 996
Thread-999 set num to 997
Thread-1000 set num to 998
Thread-998 set num to 999
Thread-997 set num to 1000
1000

可以看出，并不是按照线程的创建时间获取数据的处理权限，但是结果却可以保证每个线程都能依次处理数据，从而保证数据被自增了1000次。