本文详细介绍了Varnish3.0.0版本的更新内容及如何使用Varnish Administration Console进行管理，包括命令使用、配置调整、缓存策略和清理技术。通过实际操作示例，展示了如何优化Varnish与后端服务器以及客户端之间的通信，以及如何监控和调整缓存性能。
Varnish 3.0.0 Updates
Varnish Administration Console中的purge命令变为ban了.
help
200
help [command]
ping [timestamp]
auth response
quit
banner
status
start
stop
vcl.load <configname> <filename>
vcl.inline <configname> <quoted_VCLstring>
vcl.use <configname>
vcl.discard <configname>
vcl.list
vcl.show <configname>
param.show [-l] [<param>]
param.set <param> <value>
panic.show
panic.clear
storage.list
ban.url <regexp>
ban <field> <operator> <arg> [&& <field> <oper> <arg>]...
ban.list
另外3.0.0还改善了 varnishstat -x 的XML输出,增加了几个字段.方便Web输出.用于统计Varnish的工作状况
依赖库
Debian / Ubuntu
- autotools-dev
- automake1.9
- libtool
- autoconf
- libncurses-dev
- xsltproc
- groff-base
- libpcre3-dev
- pkg-config
Red Hat / Centos
- automake
- autoconf
- libtool
- ncurses-devel
- libxslt
- groff
- pcre-devel
- pkgconfig
安装
6月17号发布的Varnish 3.0
wget http://repo.varnish-cache.org/source/varnish-3.0.0.tar.gz
tar zxf varnish-3.0.0.tar.gz
cd varnish-3.0.0
./autogen.sh
./configure --prefix=/opt/install/
make
make install
基本配置
添加后端服务器
vi /usr/local/varlish/etc/varnish/default.vcl# nginx server backend default { .host = "127.0.0.1"; .port = "8080"; }
启动绑定80端口,
# varnishd -f /etc/varnish/default.vcl -s malloc,10M -T 127.0.0.1:2000 -a 0.0.0.0:80
使Varnish能够工作的基本配置为
backend default {
.host = "127.0.0.1";
.port = "8080";
}
使用命令 varnishlog -c 查看varnish和客户端之间的通信
使用命令 varnishlog -b 查看varnish和nginx,apache,lighttpd或其他后端web服务器之间的通信
一次前后端事务的处理过程日志
%前端事务日志 通过 varnishlog -c 获取11 SessionOpen c 220.112.96.207 43028 0.0.0.0:80 11 ReqStart c 220.112.96.207 43028 438114609 11 RxRequest c GET 11 RxURL c /js/jquery.js 11 RxProtocol c HTTP/1.1 11 RxHeader c Host: www.hezhiqiang.info 11 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 11 RxHeader c Accept-Language: zh-cn,zh;q=0.5 11 RxHeader c Accept-Encoding: gzip, deflate 11 RxHeader c Accept-Charset: UTF-8,* 11 RxHeader c Keep-Alive: 115 11 RxHeader c Connection: keep-alive 11 RxHeader c Cookie: __utma=182915519.64774331.1306435455.1308157144.1308161010.128; __utmz=182915519.1307952062.120.12.utmcsr=m437.mail.qq.com|utmccn=(referral)|utmcmd=referral|utmcct=/cgi-bin/readmail; toggle_log_detail=1; toggle_discuss_detail=1; txp_remember=0; to 11 RxHeader c If-Modified-Since: Fri, 11 Apr 2008 14:18:26 GMT 11 RxHeader c Cache-Control: max-age=0 11 VCL_call c recv 11 VCL_return c pass 11 VCL_call c hash 11 VCL_return c hash 11 VCL_call c pass 11 VCL_return c pass 11 Backend c 17 default default 11 TTL c 438114609 RFC 43200 1308164383 0 0 43200 0 11 VCL_call c fetch 11 VCL_return c pass 11 ObjProtocol c HTTP/1.1 11 ObjStatus c 304 11 ObjResponse c Not Modified 11 ObjHeader c Server: nginx/1.0.4 11 ObjHeader c Date: Wed, 15 Jun 2011 18:59:43 GMT 11 ObjHeader c Last-Modified: Fri, 11 Apr 2008 14:18:26 GMT 11 ObjHeader c Expires: Thu, 16 Jun 2011 06:59:43 GMT 11 ObjHeader c Cache-Control: max-age=43200 11 VCL_call c deliver 11 VCL_return c deliver 11 TxProtocol c HTTP/1.1 11 TxStatus c 304 11 TxResponse c Not Modified 11 TxHeader c Server: nginx/1.0.4 11 TxHeader c Last-Modified: Fri, 11 Apr 2008 14:18:26 GMT 11 TxHeader c Expires: Thu, 16 Jun 2011 06:59:43 GMT 11 TxHeader c Cache-Control: max-age=43200 11 TxHeader c Date: Wed, 15 Jun 2011 18:59:43 GMT 11 TxHeader c X-Varnish: 438114609 11 TxHeader c Age: 0 11 TxHeader c Via: 1.1 varnish 11 TxHeader c Connection: keep-alive 11 Length c 0 11 ReqEnd c 438114609 1308164383.621612310 1308164383.622174740 0.000138760 0.000510454 0.000051975 11 Debug c "herding" 11 SessionClose c timeout 11 StatSess c 220.112.96.207 43028 0 1 1 0 1 1 275 0%后端事务, 通过 varnishlog -b 获取12 BackendOpen b default 127.0.0.1 59741 127.0.0.1 8080 12 TxRequest b GET 12 TxURL b /js/jquery.js 12 TxProtocol b HTTP/1.1 12 TxHeader b Host: www.hezhiqiang.info 12 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 12 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 TxHeader b Accept-Language: zh-cn,zh;q=0.5 12 TxHeader b Accept-Encoding: gzip, deflate 12 TxHeader b Accept-Charset: UTF-8,* 12 TxHeader b Cookie: __utma=182915519.64774331.1306435455.1308157144.1308161010.128; 12 TxHeader b Pragma: no-cache 12 TxHeader b Cache-Control: no-cache 12 TxHeader b X-Forwarded-For: 220.112.96.207 12 TxHeader b X-Varnish: 438114610 12 RxProtocol b HTTP/1.1 12 RxStatus b 200 12 RxResponse b OK 12 RxHeader b Server: nginx/1.0.4 12 RxHeader b Date: Wed, 15 Jun 2011 19:01:00 GMT 12 RxHeader b Content-Type: application/x-javascript 12 RxHeader b Last-Modified: Fri, 11 Apr 2008 14:18:26 GMT 12 RxHeader b Transfer-Encoding: chunked 12 RxHeader b Connection: keep-alive 12 RxHeader b Expires: Thu, 16 Jun 2011 07:01:00 GMT 12 RxHeader b Cache-Control: max-age=43200 12 RxHeader b Content-Encoding: gzip 12 Fetch_Body b 3 0 1 12 Length b 18809 12 BackendReuse b default
查看Varnish的命中和丢失情况
varnishhist
竖线代表命中,#代表丢失
观测Varnish和后端服务器, 以及和客户端的通信
首先打开一个终端, 监听varnish和后端服务器的日志
varnishlog -b
再,打开另一个终端,监听varnish和客户端之间的通信日志
varnishlog -c
还可以再打开一个终端查看varnish的实时统计
varnishstat 0+00:14:55 Hitrate ratio: 1 1 1 Hitrate avg: 0.9181 0.9181 0.918171 0.00 0.08 Client connections accepted 311 0.00 0.35 Client requests received 258 0.00 0.29 Cache hits 23 0.00 0.03 Cache misses ....root-94:~# varnishlog -i txurl 11 TxURL b /articles/179/using-varnish-cache 11 TxURL b /css.php?n=awesome_topic 11 TxURL b / 11 TxURL b /articles/177/useing-ttyrec-and-jsttyplay-on-the-web 11 TxURL b /artifacts/jsttyplay/test.html 11 TxURL b /tag/bug/ 11 TxURL b /tag/erlang/ 11 TxURL b /tag/erlang-web/上面的输出看出,ngxin实际上只处理了动态内容.静态内容的js,css,以及图片统统被varnish包干了. nginx的工作就轻松多了.
使用Varnish Administration Console管理Varnish
带管理控制台启动 varnish
varnishd -f /usr/local/varnish/etc/varnish/default.vcl -s malloc,50M -T 127.0.0.1:2000 -a 0.0.0.0:80
登录管理控制台
root-94:~# telnet localhost 2000 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. 200 202 ----------------------------- Varnish Cache CLI 1.0 ----------------------------- Linux,2.6.39-x86_64-linode18,x86_64,-smalloc,-hcritbitType 'help' for command list. Type 'quit' to close CLI session.% 显示帮助help 200 377 help [command] ping [timestamp] auth response quit banner status start stop stats vcl.load <configname> <filename> vcl.inline <configname> <quoted_VCLstring> vcl.use <configname> vcl.discard <configname> vcl.list vcl.show <configname> param.show [-l] [<param>] param.set <param> <value> purge.url <regexp> purge <field> <operator> <arg> [&& <field> <oper> <arg>]... purge.list% 显示配置列表vcl.list 200 45 available 0 boot active 7 new% 加载配置vcl.load default /usr/local/varnish/etc/varnish/default.vcl 200 13 VCL compiled.% 切换配置vcl.use default 200 0% 删除旧的配置vcl.discard new 200 0
如果缓存命中率没有降低到不可忍受的程度,如果访问的资源没有填满varnish的cache,恭喜你,varnish运行良好.
关于缓存的Purge (过期清理)
- 为什么要手工清理缓存?
因为当我们修改了某个已经缓存的对象时,需要马上让他起作用,所以要手工清理缓存而不是等到缓存对象的TTL过期.
- 清理缓存的两种模式
- Varnish的清理策略(by lookup)
Purging by lookup 本质上是 Squid 模式, 一个特殊的 PURGE HTTP事务, 执行一次缓存查询,如果找到查询的对象,使之失效. - Varnish的清理策略(by ban list)
Varnish动则4G,8G乃至更高的缓存内容,当你要 ban 某个对象的时候,在上亿对象的Cache中去一个一个查询是低效,而且不合理的.因此当我们执行一次 ban 操作时,Varnish不是马上删除Cache中的对象,而是把这次 ban 操作放到一个叫做 ban list 的列表中,当处理从客户端来的请求时,如果缓存命中,Varnish会对比 ban list 中的对象,如果匹配则立即把Cache中的该对象TTL设置为0,强制其过期,并同时删除ban list中的这个对象.由此看来ban list 长度永远保持在一个比较小的状态.即使是在内容更新频繁的巨型网站, 十万百万级的ban list长度和有上亿,十亿对象的Cache相比也是小巫见大巫,不足惧也. 原文 Varnish purges 第二节 How Varnish attacks the problem 第一段很清楚的说明了Varnish的purge策略.
- Varnish的清理策略(by lookup)
- 为什么要用程序处理Varnish purge操作
扫一扫
578
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
