Easy to use object-oriented thread pool framework

A thread pool is an object that maintains a pool of worker threads to perform time consuming operations in parallel. It assigns jobs to the threads by putting them in a work request queue, where they are picked up by the next available thread. This then performs the requested operation in the background and puts the results in another queue.

The thread pool object can then collect the results from all threads from this queue as soon as they become available or after all threads have finished their work. It’s also possible, to define callbacks to handle each result as it comes in.

DEMO
#!/usr/bin/env python
# coding=utf8
# author=ff0000team

import threadpool as tp

def test(url, passwd):
    print url, passwd

if __name__ == '__main__':
    args = [
        (['http://xxx.com', '123'], {}),
        (['http://yyy.com', '213'], {}), 
        (['http://zzz.com', '321'], {})
    ]
    '''
    ``args_list`` contains the parameters for each invocation of callable.
    Each item in ``args_list`` should be either a 2-item tuple of the list of
    positional arguments and a dictionary of keyword arguments or a single,
    non-tuple argument.
    '''
    pool = tp.ThreadPool(2)
    reqs = tp.makeRequests(test, args)
    [pool.putRequest(req) for req in reqs]
    pool.wait()


除此之外，map()函数也可以简化多线程。
http://segmentfault.com/blog/caspar/1190000000414339
map 这一小巧精致的函数是简捷实现 Python 程序并行化的关键。map 源于 Lisp 这类函数式编程语言。它可以通过一个序列实现两个函数之间的映射。

在 Python 中有个两个库包含了 map 函数： multiprocessing 和它鲜为人知的子库 multiprocessing.dummy.

import urllib2 
from multiprocessing.dummy import Pool as ThreadPool 

urls = [
    'http://www.python.org', 
    'http://www.python.org/about/',
    'http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html',
    'http://www.python.org/doc/',
    'http://www.python.org/download/',
    'http://www.python.org/getit/',
    'http://www.python.org/community/',
    'https://wiki.python.org/moin/',
    'http://planet.python.org/',
    'https://wiki.python.org/moin/LocalUserGroups',
    'http://www.python.org/psf/',
    'http://docs.python.org/devguide/',
    'http://www.python.org/community/awards/'
    # etc.. 
    ]

# Make the Pool of workers
pool = ThreadPool(4) 
# Open the urls in their own threads
# and return the results
results = pool.map(urllib2.urlopen, urls)
#close the pool and wait for the work to finish 
pool.close() 
pool.join()