Using varnish to accelerate file serving out of GridFS

最新推荐文章于 2025-01-14 23:49:51 发布

转载最新推荐文章于 2025-01-14 23:49:51 发布 · 903 阅读

mongodb 同时被 2 个专栏收录

126 篇文章

订阅专栏

performance

47 篇文章

订阅专栏

通过将Varnish部署为HTTP反向代理，可以在前端缓存静态文件，显著提高从GridFS读取文件的速度，减少延迟，并优化并发请求处理能力。此外，文章还详细介绍了如何避免在生产环境中因Cookie导致的缓存不适用问题，并提供了配置示例。实验证明，使用Varnish缓存后，文件服务性能接近直接从文件系统读取，且大大提高了整体效率。

After migrating some image files from file system to gridfs and serving them using nginx-gridfs, I realized that serving files out of gridfs is significantly slower than before (This is also stated in this article.). Even worse, since there is no non-blocking implementation of mongodb c driver, fetching single file from gridfs blocks nginx worker from accepting other requests. So it’s necessary to minimize gridfs queries by using caches.

Caching in front of web server

Varnish is a caching HTTP reverse proxy, can be installed in front of any server that speaks HTTP and configured to cache the contents.

Below is a brief result of running a reply-rate benchmark on four different configurations:

Requests/second	without varnish	with varnish
filesystem	392.34	641.98
GridFS	161.95	646.54

From the “without varnish” column we can discover that, serving files from GridFS degrades the performance by 58.7% compared to serving directly from file system. But after setting up the cache, they performs almost the same. Further more, they both take the great benefit of the cache acceleration, which makes it quite fast for serving static files.

Be careful of Cookies

When taking cache proxy into production, things are not always ideal. By default, varnish will not process a request as cacheable if there is a cookie header. Use a cookie-free domain for static files if possible.

In some circumstance you may want to ignore particular cookies, this can be done by adding some code to the vcl_recv section of the configuration file, like:

if (req.http.Cookie) {
    set req.http.Cookie = regsuball(req.http.Cookie, "(^|; ) *__utm.=[^;]+;? *", "\1");
    if (req.http.Cookie == "") {
        remove req.http.Cookie;
    }
}

Appendix: Raw Results of Apache Benchmark

The benchmark runs on a Virtualbox virtual machine, with single CPU core, 512MB RAM and 8GB disk.

Read filesystem, without varnish cache:

$ ab -n10000 -c10 http://192.168.1.183:8080/fs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
 
Server Software:        nginx/1.2.0
Server Hostname:        192.168.1.183
Server Port:            8080
 
Document Path:          /fs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
Document Length:        32095 bytes
 
Concurrency Level:      10
Time taken for tests:   25.488 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      323790000 bytes
HTML transferred:       320950000 bytes
Requests per second:    392.34 [#/sec] (mean)
Time per request:       25.488 [ms] (mean)
Time per request:       2.549 [ms] (mean, across all concurrent requests)
Transfer rate:          12405.81 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:     4   25   9.6     22     111
Waiting:        2   23   8.5     20      97
Total:          4   25   9.5     22     112
 
Percentage of the requests served within a certain time (ms)
  50%     22
  66%     23
  75%     25
  80%     27
  90%     36
  95%     48
  98%     57
  99%     62
 100%    112 (longest request)

Read GridFS, without varnish cache:

$ ab -n10000 -c10 http://192.168.1.183:8080/gridfs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
 
Server Software:        nginx/1.2.0
Server Hostname:        192.168.1.183
Server Port:            8080
 
Document Path:          /gridfs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
Document Length:        32095 bytes
 
Concurrency Level:      10
Time taken for tests:   61.748 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      323990000 bytes
HTML transferred:       320950000 bytes
Requests per second:    161.95 [#/sec] (mean)
Time per request:       61.748 [ms] (mean)
Time per request:       6.175 [ms] (mean, across all concurrent requests)
Transfer rate:          5123.97 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       6
Processing:    11   61  25.3     58     696
Waiting:       10   54  20.3     54     637
Total:         11   62  25.3     58     696
 
Percentage of the requests served within a certain time (ms)
  50%     58
  66%     63
  75%     68
  80%     71
  90%     84
  95%     96
  98%    106
  99%    115
 100%    696 (longest request)

Read GridFS, with varnish cache:

$ ab -n10000 -c10 http://192.168.1.183/gridfs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
 
Server Software:        nginx/1.2.0
Server Hostname:        192.168.1.183
Server Port:            80
 
Document Path:          /gridfs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
Document Length:        32095 bytes
 
Concurrency Level:      10
Time taken for tests:   15.467 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      324840000 bytes
HTML transferred:       320950000 bytes
Requests per second:    646.54 [#/sec] (mean)
Time per request:       15.467 [ms] (mean)
Time per request:       1.547 [ms] (mean, across all concurrent requests)
Transfer rate:          20509.98 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  30.0      0    1003
Processing:     1   14   7.9     13      78
Waiting:        0   11   6.2     10      75
Total:          1   15  30.9     13    1023
 
Percentage of the requests served within a certain time (ms)
  50%     13
  66%     16
  75%     19
  80%     20
  90%     25
  95%     28
  98%     34
  99%     39
 100%   1023 (longest request)

Read filesystem, with varnish cache:

$ ab -n10000 -c10 http://192.168.1.183/fs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
 
Server Software:        nginx/1.2.0
Server Hostname:        192.168.1.183
Server Port:            80
 
Document Path:          /fs/image/2012/05/11/small_e3491035-6bb5-469f-a20c-eaa9f2ec7e3f.jpg
Document Length:        32095 bytes
 
Concurrency Level:      10
Time taken for tests:   15.577 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      324420000 bytes
HTML transferred:       320950000 bytes
Requests per second:    641.98 [#/sec] (mean)
Time per request:       15.577 [ms] (mean)
Time per request:       1.558 [ms] (mean, across all concurrent requests)
Transfer rate:          20339.06 [Kbytes/sec] received
 
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2  43.6      0    1003
Processing:     1   13   8.4     12     109
Waiting:        0   10   6.8     10      90
Total:          1   15  44.4     12    1033
 
Percentage of the requests served within a certain time (ms)
  50%     12
  66%     16
  75%     18
  80%     20
  90%     24
  95%     28
  98%     33
  99%     38
 100%   1033 (longest request)