实现并发的手段
进程:启动多个进程,进程之间由操作系统负责调用
线程:启动多个线程,真正被CPU执行的最小单位
(a)开启一个线程,创建一个线程,寄存器,堆栈
(b)关闭一个线程
协程:
(1)本质上是一个线程
(2)能够在多个任务之间切换来节省一些IO时间
(3)协程中任务之间的切换也消耗时间,但是开销要远远小于线程和进程之间的切换
greenlet模块实现协程之间的切换
from greenlet import greenlet
import gevent
def eat():
print('esting start')
g2.switch() # 切换到g2去执行,保存这个状态
print('eating wait')
g2.switch() # 再切换到g2去
def play():
print('playing start')
g1.switch() # 切换到g1去执行
print('playing end')
g1 = greenlet(eat)
g2 = greenlet(play)
g1.switch() # 切换g1执行
注意:
(1)进程和线程的任务切换由操作系统完成
(2)协程任务之间的切换由程序代码完成,只有遇到协程模块能识别的IO操作的时候,程序才会切换,实现并发效果
有些模块gevent识别不了,如time和socket,所以需要打个补丁:
from gevent import monkey; monkey.patch_all() # 让gevent认识time模块
gevent实现自动切换(封装了greenlet),遇到IO自动切换
from gevent import monkey; monkey.patch_all() # 让gevent认识time模块
import time
import gevent # 封装了greenlet,看不见切换
import threading
def eat():
print(threading.current_thread().getName())
print('esting start')
time.sleep(1)
print('eating end')
def play():
print(threading.current_thread().getName())
print('playing start')
time.sleep(1)
print('playing end')
g1 = gevent.spawn(eat)
g2 = gevent.spawn(play)
g1.join() # 等待任务执行
g2.join()
协程异步
from gevent import monkey; monkey.patch_all()
import time
import gevent
def task():
time.sleep(1)
print('111')
#def sync():
# for i in range(10):
# task()
def async():
g_lst = []
for i in range(10):
g = gevent.spawn(task)
g_lst.append(g)
gevent.joinall(g_lst)
# sync() # 同步
async() # 异步
协程:
(1)能够在一个线程中实现并发效果的概念
(2)能够规避一些任务中的IO操作
(3)在任务执行过程中,检测到IO就切换到其他任务
(4)在一个线程上提高CPU的利用率
(5)相对于多线程,协程切换的效率更快
爬虫的例子:请求过程中的IO等待
from gevent import monkey; monkey.patch_all()
import gevent
import requests
def get(url):
response = requests.get(url)
if response.status_code == 200:
content = response.content.decode('utf-8')
return len(content)
url_l = [
'https://www.cnblogs.com/',
'http://www.baidu.com',
'http://www.sohu.com',
'https://www.sogou.com',
]
g_lst = []
for url in url_l:
g = gevent.spawn(get, url)
g_lst.append(g)
gevent.joinall(g_lst)
for g in g_lst:
print(g.value)
协程实现socket
s端
from gevent import spawn, monkey; monkey.patch_all()
import socket
def server(ip, port):
print('===========')
sk = socket.socket()
sk.bind((ip, port))
sk.listen(5)
while True:
conn, addr = sk.accept()
spawn(task, conn)
sk.close()
def task(conn):
while True:
res = conn.recv(1024)
if not res: break
conn.send(res.upper())
if __name__ == '__main__':
g = spawn(server, '127.0.0.1', 8080)
g.join()
c端(模拟几百个c端去链接s端)
import socket
from threading import Thread, current_thread
def client(ip, port):
sk = socket.socket()
sk.connect((ip, port))
while True:
sk.send(('%s hello' % current_thread().getName()).encode('utf-8'))
res = sk.recv(1024)
print(res.decode('utf-8'))
if __name__ == '__main__':
for i in range(100):
t = Thread(target=client, args=('127.0.0.1', 8080))
t.start()