Docker 下 【Locust 2.8】 + prometheus + influxdb + grafana 性能测试结果采集、监控、数据持久化

本文使用docker部署，建议使用docker-compose一键部署方式，观看这篇：
https://blog.csdn.net/qq_41522024/article/details/128997583

环境准备

1. Docker + Docker Compose 环境已配置好
2. 打包好你的镜像，并且镜像中已经包含了你的测试代码

性能测试结果采集代码

网上 99.99% 的教程都是用这个：https://github.com/myzhan/boomer/blob/master/prometheus_exporter.py

当然我也不例外 😛

需要注意的是，2.8.3 版本，在分布式运行的时候，需要你在 master 以及 worker 都要有相同的采集代码

# coding: utf8

import six
from itertools import chain

from flask import request, Response
from locust import stats as locust_stats, runners as locust_runners
from locust import User, task, events
from prometheus_client import Metric, REGISTRY, exposition

# This locustfile adds an external web endpoint to the locust master, and makes it serve as a prometheus exporter.
# Runs it as a normal locustfile, then points prometheus to it.
# locust -f prometheus_exporter.py --master

# Lots of code taken from [mbolek's locust_exporter](https://github.com/mbolek/locust_exporter), thx mbolek!


class LocustCollector(object):
    registry = REGISTRY

    def __init__(self, environment, runner):
        self.environment = environment
        self.runner = runner

    def collect(self):
        # collect metrics only when locust runner is spawning or running.
        runner = self.runner

        if runner and runner.state in (locust_runners.STATE_SPAWNING, locust_runners.STATE_RUNNING):
            stats = []
            for s in chain(locust_stats.sort_stats(runner.stats.entries), [runner.stats.total]):
                stats.append({
                    "method": s.method,
                    "name": s.name,
                    "num_requests": s.num_requests,
                    "num_failures": s.num_failures,
                    "avg_response_time": s.avg_response_time,
                    "min_response_time": s.min_response_time or 0,
                    "max_response_time": s.max_response_time,
                    "current_rps": s.current_rps,
                    "median_response_time": s.median_response_time,
                    "ninetieth_response_time": s.get_response_time_percentile(0.9),
                    # only total stats can use current_response_time, so sad.
                    # "current_response_time_percentile_95": s.get_current_response_time_percentile(0.95),
                    "avg_content_length": s.avg_content_length,
                    "current_fail_per_sec": s.current_fail_per_sec
                })

            # perhaps StatsError.parse_error in e.to_dict only works in python slave, take notices!
            errors = [e.to_dict() for e in six.itervalues(runner.stats.errors)]

            metric = Metric('locust_user_count', 'Swarmed users', 'gauge')
            metric.add_sample('locust_user_count', value=runner.user_count, labels={})
            yield metric

            metric = Metric('locust_errors', 'Locust requests errors', 'gauge')
            for err in errors:
                metric.add_sample('locust_errors', value=err['occurrences'],
                                  labels={'path': err['name'], 'method': err['method'],
                                          'error': err['error']})
            yield metric

            is_distributed = isinstance(runner, locust_runners.MasterRunner)
            if is_distributed:
                metric = Metric('locust_slave_count', 'Locust number of slaves', 'gauge')
                metric.add_sample('locust_slave_count', value=len(runner.clients.values()), labels={})
                yield metric

            metric = Metric('locust_fail_ratio', 'Locust failure ratio', 'gauge')
            metric.add_sample('locust_fail_ratio', value=runner.stats.total.fail_ratio, labels={})
            yield metric

            metric = Metric('locust_state', 'State of the locust swarm', 'gauge')
            metric.add_sample('locust_state', value=1, labels={'state': runner.state})
            yield metric

            stats_metrics = ['avg_content_length', 'avg_response_time', 'current_rps', 'current_fail_per_sec',
                             'max_response_time', 'ninetieth_response_time', 'median_response_time',
                             'min_response_time',
                             'num_failures', 'num_requests']

            for mtr in stats_metrics:
                mtype = 'gauge'
                if mtr in ['num_requests', 'num_failures']:
                    mtype = 'counter'
                metric = Metric('locust_stats_' + mtr, 'Locust stats ' + mtr, mtype)
                for stat in stats:
                    # Aggregated stat's method label is None, so name it as Aggregated
                    # locust has changed name Total to Aggregated since 0.12.1
                    if 'Aggregated' != stat['name']:
                        metric.add_sample('locust_stats_' + mtr, value=stat[mtr],
                                          labels={'path': stat['name'], 'method': stat['method']})
                    else:
                        metric.add_sample('locust_stats_' + mtr, value=stat[mtr],
                                          labels={'path': stat['name'], 'method': 'Aggregated'})
                yield metric


# prometheus监听端口
@events.init.add_listener
def locust_init(environment, runner, **kwargs):
    print("locust init event received")
    if environment.web_ui and runner:
        @environment.web_ui.app.route("/export/prometheus")
        def prometheus_exporter():
            registry = REGISTRY
            encoder, content_type = exposition.choose_encoder(request.headers.get('Accept'))
            if 'name[]' in request.args:
                registry = REGISTRY.restricted_registry(request.args.get('name[]'))
            body = encoder(registry)
            return Response(body, content_type=content_type)

        REGISTRY.register(LocustCollector(environment, runner))

使用方式：

# 可以是以模块的方式导入你的 locustfile 中
from prometheus_exporter import *

运行脚本之后，访问http://localhost:8089/export/prometheus 测试一下，如图输出信息即为成功

prometheus + influxdb + grafana

prometheus influxdb配置

这个时候，你的 locust 镜像应该已经创建好了，镜像里面也应该包含 采集代码 以及的 测试脚本

运行 prometheus + influxdb 的 docker-compose-prometheus-influxdb.yml

docker-compose.yml 包含了 prometheus 、 influxdb 、 locust_master 的容器

docker-compose -f /your/path/docker-compose-prometheus-influxdb.yml up -d

控制台会输出：

Creating influxdb … done
Creating locust_master … done
Creating prometheus … done

这个时候，访问http://IP:9090/targets ，如图 prometheus + influxdb 的配置即为成功

因为 docker-compose.yml 中已经配置好了卷的映射，关闭容器或者重启容器数据都不会丢失

grafana 的配置

1. 启动容器

   docker run -d -p 3000:3000 --name grafana grafana/grafana
   

2. 访问 http://IP:3000/

   初始账户密码：admin/admin

3. 添加数据源，选择prometheus

4. 输入prometheus的IP以及端口

5. 点击 import，选择模板：12081

6. 导入刚刚创建的数据源

7. 查看仪表盘

8. 执行测试之后的结果：