Python核心编程第四章总结

最新推荐文章于 2022-04-20 12:46:29 发布

原创最新推荐文章于 2022-04-20 12:46:29 发布 · 186 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#python #多线程 #thread #多进程

python_core_program 专栏收录该内容

2 篇文章

订阅专栏

本文通过具体示例介绍Python多线程编程技巧，包括使用thread和threading模块创建线程，实现资源共享，解决同步问题，并演示了生产者与消费者的模式。

Python核心编程

第四章总结（I/O密集型使用多线程，计算密集型使用多进程）

time.sleep()需要一个浮点型参数来指明这个程序会暂停指定的时间。
thread模块

start_new_thread(function, args, kwargs = None)
派生一个新的线程，参数包括函数名，函数的参数，和可选的关键字参数。即使要执行的函数不需要参数，也需要传递一个空元组。
allocate_lock()
分配LockType锁对象
exit（）
退出线程

LockType锁对象的方法

acquire（wait = None)
尝试获取锁对象
locked（）
如果获取了锁对象则返回True，否则返回False
release（）
释放锁

import thread
from time import sleep, ctime

def loop0():
    print('start loop 0 at:', ctime())
    sleep(4)
    print('loop 0 done at:', ctime())

def loop1():
    print('start loop 1 at:', ctime())
    #thread.start_new_thread(loop3,())
    sleep(2)
    print('loop 1 done at:', ctime())
    
'''
def loop3():
    print('start loop 3 at:',ctime())
    sleep(1)
    print('loop 3 done at:', ctime())
'''

def main():
    print('starting at:', ctime())
    thread.start_new_thread(loop0,())
    thread.start_new_thread(loop1,())
    sleep(6)
    print('all DONE at:', ctime())

if __name__ == '__main__':
    main()

starting at: Sun Mar 21 21:00:05 2021
start loop 0 at: Sun Mar 21 21:00:05 2021
start loop 1 at: Sun Mar 21 21:00:05 2021
loop 1 done at: Sun Mar 21 21:00:07 2021
loop 0 done at: Sun Mar 21 21:00:09 2021
all DONE at: Sun Mar 21 21:00:11 2021

主线程之所以有一个sleep（6）是因为如果没有，主线程执行完语句之后会退出，而loop0和loop1会直接终止。

此时我们可以添加锁对象而不必额外延时6秒。先为每个线程获取锁对象，判断每个线程是否释放了锁，释放则结束。

import thread
from time import sleep, ctime

loops = [4, 2]

def loop(nloop, nsec, lock):
    print('start loop', nloop, 'at:', ctime())
    sleep(nsec)
    print('loop', nloop, 'done at:', ctime())
    lock.release()

def main():
    print('starting at:', ctime())
    locks = []
    nloops = range(len(loops))

    for i in nloops:
        lock = thread.allocate_lock()       #获取锁对象
        lock.acquire()                      #把锁锁上
        locks.append(lock)

    for i in nloops:
        thread.start_new_thread(loop,(i, loops[i], locks[i]))

    for i in nloops:
        while locks[i].locked():pass
        
    print('all DONE at:', ctime())

if __name__ == '__main__':
    main()

starting at: Sun Mar 21 21:10:53 2021
start loop 0 at: Sun Mar 21 21:10:53 2021
start loop 1 at: Sun Mar 21 21:10:53 2021
loop 1 done at: Sun Mar 21 21:10:55 2021
loop 0 done at: Sun Mar 21 21:10:57 2021
all DONE at: Sun Mar 21 21:10:57 2021

threading模块

Thread类

属性：

name	线程名
ident	线程的标识符
daemon	布尔标志，表示这个线程是否为守护线程

方法：

__init __	实例化一个线程对象
start（）	开始执行该线程
run（）	定义线程功能的方法（通常在子类中重写）
–	–
join（timeout = None）	直至启动的线程终止之前一直挂起；除非给出了timeout（秒），否则一直阻塞，用于判断线程是否终止

import threading
from time import sleep, ctime
loops = [4, 2]

def loop(nloop, nsec):
    print('start loop', nloop, 'at:', ctime())
    sleep(nsec)
    print('loop', nloop, 'done at:', ctime())

def main():
    print('starting at:', ctime())
    threads = []
    nloops = range(len(loops))

    for i in nloops:
        t = threading.Thread(target=loop, args = (i, loops[i]))
        threads.append(t)
    
    for i in nloops:
        threads[i].start()

    for i in nloops:
        threads[i].join()

    print('all DONE at:', ctime())

if __name__ == '__main__':
    main()

starting at: Sun Mar 21 21:29:39 2021
start loop 0 at: Sun Mar 21 21:29:39 2021
start loop 1 at: Sun Mar 21 21:29:39 2021
loop 1 done at: Sun Mar 21 21:29:41 2021
loop 0 done at: Sun Mar 21 21:29:43 2021
all DONE at: Sun Mar 21 21:29:43 2021

这里是创建Thread的实例，将函数和参数传递给它。

import threading
from time import sleep, ctime
loops = [4, 2]

class ThreadFunc:
    def __init__(self, func, args, name = ''):
        self.func = func
        self.args = args
        self.name = name

    def __call__(self):
        self.func(*self.args)

def loop(nloop, nsec):
    print('start loop', nloop, 'at:', ctime())
    sleep(nsec)
    print('loop', nloop, 'done at:', ctime())

def main():
    print('starting at:', ctime())
    threads = []
    nloops = range(len(loops))

    for i in nloops:
        t = threading.Thread(target=ThreadFunc(loop, (i, loops[i]), loop.__name__))
        threads.append(t)
    
    for i in nloops:
        threads[i].start()

    for i in nloops:
        threads[i].join()

    print('all DONE at:', ctime())

if __name__ == '__main__':
    main()

这是传入一个可调用的实例。

import threading
from time import sleep, ctime

loops = (4, 2)

class MyThread(threading.Thread):
    def __init__(self, func, args, name = ''):
        threading.Thread.__init__(self)
        self.name = name
        self.func = func
        self.args = args
    
    def run(self):
        self.func(*self.args)

def loop(nloop, nsec):
    print('start loop', nloop, 'at:', ctime())
    sleep(nsec)
    print('loop', nloop, 'done at:', ctime())

def main():
    print('starting at:', ctime())
    threads = []
    nloops = range(len(loops))
    
    for i in nloops:
        t = MyThread(loop, (i, loops[i]), loop.__name__)
        threads.append(t)

    for i in nloops:
        threads[i].start()

    for i in nloops:
        threads[i].join()

    print('all DONE at:', ctime())

if __name__ == '__main__':
    main()

这是创建一个Thread的子类，在子类的run函数定义这个线程做什么。

from atexit import register
@register
def fun():
	pass

这是一个装饰器，fun（）会在程序退出之前执行，因此不必设置join。

2to3工具

找到2to3.py。。。。。。

同步问题

锁机制

当多个线程争夺锁时，允许第一个获得锁的线程进入临界区，并执行代码。之后到达的线程都会被阻塞，知道第一个线程执行结束，退出临界区，并释放锁。此时其他等待的线程可以获得锁并进入临界区，不过这些阻塞的线程谁先获得锁是不确定的（不是先到先得）。

print('%s' % (a or b))

先创建一个锁对象
当需要使用共享资源时，先lock.acquire()，使用完之后，lock.release()。也可以with lock，然后使用共享资源，无需释放。

信号量

from atexit import register
from random import randrange
from threading import Thread, BoundedSemaphore, Lock
from time import sleep, ctime

lock = Lock()
MAX = 5
candytray = BoundedSemaphore(MAX)

def refill():

    lock.acquire()
    print('Refilling candy...')
    try:
        candytray.release()
    except ValueError:
        print('full, skipping')
    else:
        print('OK')
    lock.release()

def buy():
    lock.acquire()
    print('Buying candy...')
    if candytray.acquire(False):
        print('OK')
    else:
        print('empty, skipping')
    lock.release()

def producer(loops):
    for i in range(loops):
        refill()
        sleep(randrange(3))

def consumer(loops):
    for i in range(loops):
        buy()
        sleep(randrange(3))

def _main():
    print('starting at:', ctime())
    nloops = randrange(2, 6)
    print('The candy machine (full with %d bars)!' % MAX)
    Thread(target=consumer, args=(randrange(nloops,nloops+MAX+2),)).start()
    Thread(target=producer,args=(nloops,)).start()

@register
def _atexit():
    print('all done at:', ctime())

if __name__ == '__main__':
    _main()

信号量的P，V操作在python中也是acquire（）和release（）。BoundedSemaphore保证资源量不会超过初始值。

生产者与消费者和queue模块

from random import randint
from time import sleep
from queue import Queue
from myThread import MyThread

def writeQ(queue):
    print('producing object for Q...')
    queue.put('xxx',1)				
    print('size now', queue.qsize())

def readQ(queue):
    val = queue.get(1)
    print('consumed object from Q... size now', queue.qsize())

def writer(queue, loops):
    for i in range(loops):
        writeQ(queue)
        sleep(randint(1, 3))

def reader(queue, loops):
    for i in range(loops):
        readQ(queue)
        sleep(randint(2, 5))

funcs = [writer, reader]
nfuncs = range(len(funcs))

def main():
    nloops = randint(2, 5)
    q = Queue(32)

    threads = []
    for i in nfuncs:
        
        t = MyThread(funcs[i], (q, nloops), funcs[i].__name__)
        threads.append(t)

    for i in nfuncs:
        threads[i].start()

    for i in nfuncs:
        threads[i].join()

    print('all Done')

if __name__ == '__main__':
    main()