python web一锅烩（flask,gunicom,gevent,tornado）

最新推荐文章于 2024-05-27 01:37:22 发布

laputa73

最新推荐文章于 2024-05-27 01:37:22 发布

阅读量1.4k

点赞数

CC 4.0 BY-SA版权

分类专栏： python

本文链接：https://blog.youkuaiyun.com/laputa73/article/details/79174884

python 专栏收录该内容

27 篇文章

订阅专栏

本文探讨了Python中WSGI协议以及基于该协议的Web框架Flask、独立WSGI服务器Gunicorn、非阻塞协程库Gevent和异步框架Tornado的组合使用。通过各种组合测试，如Flask与Gevent、Gunicorn的结合，最终确定推荐的Python Web解决方案为Flask+Gevent+Gunicorn。尽管性能相较于Perl的某些框架较低，但在实际应用中，它们的性能差距会缩小。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

先说几点结论：

WSGI是webserver和webapp的接口协议，相当于以前的CGI。

flask是一个支持WSGI的web框架，同时自带了WSGI server. 相当于perl的mojo.

gunicorn是一个独立的WSGI server(其他的都是模块)。可以支持gevent.支持woker模式。相当于nginx.

gevent是python的非阻塞协程框架，包含了WSGIserver。相当于perl的coro,

tornado是python的非阻塞异步框架，包含了WSGI server和web框架。相当于perl的AE.

所以，会有以下这些组合。

1.纯的flask web app

#!/bin/env python
# -*- coding: utf-8 -*-
from flask import Flask,render_template

app = Flask(__name__)

@app.route('/')
def index():
    return "Hello, World!"

@app.route('/sleep')
def main_handler():
    #gevent.sleep(0.5);
    return "sleep a while!"
    #return render_template('main_j2.html', messages="whatever",title="home")
    
    
if __name__ == '__main__':
#    app.run(debug=True)    #默认5000
    app.run(port=5000, debug=False)


#最简单的flask app
#测试环境阿里云，启用virtualenv. source flask/bin/activate
#使用flask自带WSGIserver启动
# ./app.py
# * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
#测试$ curl 127.0.0.1:5000
#Hello, World!
#性能测试ab -k -n 10000 -c 10 http://127.0.0.1:5000/
#Requests per second:    829.38 [#/sec] (mean)
#Requests per second:    986.63 [#/sec] (mean)
#Requests per second:    824.06 [#/sec] (mean)
#ab -k -n 10000 -c 100 http://127.0.0.1:5000/
#Requests per second:    965.84 [#/sec] (mean)
#注：关闭debug并无明显提升

2. 这个flask应用可以用Gunicorn来启动，并且可以实现多进程，来提升性能。

#使用gunicorn
#ab -k -c 100 -n 1000 127.0.0.1:5000/
##gunicorn -w 1 -b 127.0.0.1:5000  flaskapp1:app
#Requests per second:    1085.90 [#/sec] (mean)
##gunicorn -w 2 -b 127.0.0.1:5000  flaskapp1:app
#Requests per second:    2104.63 [#/sec] (mean)
#有近一倍的提升

flask是一个同步框架,在压力比较小的情况下，性能问题不是很突出。不过大并发就撑不住了。所以还需要借助非阻塞的WSGI server.

3.用gevent WSGI server来启动flask app.

gevent需要最低python2.7,阿里云默认的2.6需要升级。

#!/bin/env python
# -*- coding: utf-8 -*-

from gevent.pywsgi import WSGIServer
from flaskapp1 import app


if __name__ == '__main__':
    WSGIServer(('127.0.0.1', 5000), app).serve_forever()
#WSGIServer(('', 5000), app).serve_forever()

#http_server = WSGIServer(('', 5000), app)
#http_server.serve_forever()


    


#flask app+gevent
#测试环境阿里云，启用virtualenv. source flask/bin/activate
#先升级pip install --upgrade pip
#再升级python到2.7
#tar -xjf Python-2.7.5.tar.bz2
#./configure --prefix=/usr/local/python-2.7.5
#
#使用gevent的WSGIserver启动
# ./flaskapp2.py
# * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
#测试$ curl 127.0.0.1:5000
#Hello, World!
#性能测试ab -k -n 10000 -c 10 http://127.0.0.1:5000/
#Requests per second:    1235.15 [#/sec] (mean)
#ab -k -n 10000 -c 100 http://127.0.0.1:5000/
#Requests per second:    1323.66 [#/sec] (mean)
#比flask有明显提升
#使用gunicorn
#ab -k -n 10000 -c 100 http://127.0.0.1:5000/
#gunicorn -w 1 -k gevent -b 127.0.0.1:5000  flaskapp3:app
#1254.49 [#/sec] (mean)   单进程差不多
#gunicorn -w 2 -k gevent -b 127.0.0.1:5000  flaskapp3:app
#Requests per second:    2392.21 [#/sec] (mean)   2进程快一倍

使用gunicorn并不能提升单进程gevent的性能，但是它提供了prefork的master-work模式，可以方便起多进程，

4.在flask + gevent.wsgi的基础上，再整合进，gevent.httpclient,形成一小型的服务框架。

#!/bin/env python
# -*- coding: utf-8 -*-

import gevent
from gevent.pywsgi import WSGIServer
from flask import Flask

from geventhttpclient import HTTPClient
from geventhttpclient.url import URL

app = Flask(__name__)

@app.route('/')
def index():
    return "Hello, World!"

@app.route('/sleep')
def sleep_handler():
    gevent.sleep(0.5);
    return "sleep a while!"

@app.route('/robot')
def robot_handler():
    n=3
    coros=range(1,n)
    urls=('http://www.baidu.com','http://www.taobao.com','http://www.sina.com.cn');
    for j in xrange(1,n):
    	coros[j-1] =  gevent.spawn(robot, urls[j-1])
#    gevent.joinall(coros)    #加这个代表3个做完了才返回
    return "robot over!"


	
def robot(urls):
	#url = URL('http://www.baidu.com/')
	url=URL(urls)
	#http = HTTPClient(url.host)
	http = HTTPClient(url.host,connection_timeout=10,network_timeout=10)

	# issue a get request
	response = http.get(url.request_uri)

	# read status_code
	#response.status_code
	print 'status_code(%s) is %s.' % (urls,response.status_code)

	# read response body
	#body = response.read()
	#print 'body is %s.' % body

	# close connections
	http.close()


if __name__ == '__main__':
    print('Serving on 5000...')
    WSGIServer(('127.0.0.1', 5000), app).serve_forever()

因为都是基于gevent,所以server和client配合很流畅。

5.再试一把flask app on tornado wsgi server

from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop

from flask import Flask




app = Flask(__name__)

@app.route('/')
def index():
    return "Hello, World!"

if __name__ == '__main__':
    print('Serving on 5000...')
    http_server = HTTPServer(WSGIContainer(app))
    http_server.listen(5000)
    IOLoop.instance().start()

性能和flask on gevent差不多

6.再试试tornado app on gevent wsgi server

import os.path
#import tornado.ioloop
import tornado.web

import tornado.wsgi
import gevent.wsgi

import gevent
    
class MainHandler(tornado.web.RequestHandler):
    def get(self):
    	  self.write("Hello, world")
    	  self.write(self.request.remote_ip)
        #self.render('main.html', page_title="", body_id="", messages="whatever",title="home")

class SleepHandler(tornado.web.RequestHandler):
    def get(self):
    	  gevent.sleep(0.5);
    	  self.write("Sleep a while")
    	  #self.write(self.request.remote_ip)
        #self.render('main.html', page_title="", body_id="", messages="whatever",title="home")
        
settings = {
    "static_path":os.path.join(os.path.dirname(__file__),'static'),
    "template_path":"templates",
}
application = tornado.wsgi.WSGIApplication([
    (r"/", MainHandler),(r"/sleep", SleepHandler)
], **settings)

#application = tornado.wsgi.WSGIApplication([
#    (r"/", pure_tornado.MainHandler),
#],**pure_tornado.settings)

if __name__ == "__main__":
    server = gevent.wsgi.WSGIServer(('127.0.0.1', 5000), application)
    server.serve_forever()

性能和flask on gevnt也差不多。说明tornado作为web框架，性能上并不具备太大优势。

另外，下面两个组合是失败的

1. flask app直接加gevent.monkey.patch_all。不能实现非阻塞。

2. flask app + tornado wsgi + gevent.httpclient. tornado的wsgi server只能配合异步的api,不能配合协程的api

基于以上测试，确定后续python的web采用flask+gevent+gunicorn的组合。

ps.python的这些web 框架性能好低。(1krps)

不像perl的feersum和anteventhttpserver，benchemark性能超强。(7k~8krps)

不过加上实际应用，就都一样下来了。