ngxtop与Grafana变量：动态切换监控目标的仪表盘配置

ngxtop与Grafana变量：动态切换监控目标的仪表盘配置

【免费下载链接】ngxtop Real-time metrics for nginx server 项目地址: https://gitcode.com/gh_mirrors/ng/ngxtop

1. 痛点与解决方案概述

在多实例Nginx环境中，传统静态监控仪表盘存在切换目标繁琐、配置冗余的问题。本文将通过ngxtop数据采集、Prometheus动态发现与Grafana变量配置的协同方案，实现"一键切换监控目标"的高效运维体验。读者将掌握完整的配置流程，包括Prometheus服务发现配置、Grafana变量定义及仪表盘模板制作。

2. 技术架构与组件关系

2.1 核心组件架构

2.2 关键文件与功能

组件	核心配置文件	功能描述
ngxtop	ngxtop/ngxtop.py	Nginx实时指标采集，支持自定义查询
Prometheus	prometheus/prometheus.yml	配置监控目标与数据抓取规则
Grafana	仪表盘JSON文件	定义变量与动态查询逻辑

3. Prometheus服务发现配置

3.1 静态配置基础版

# prometheus/prometheus.yml 第8-10行
- job_name: 'ngxtop'
  static_configs:
    - targets: ['ngxtop:8080']  # 单实例静态配置

3.2 文件服务发现进阶版

创建targets.json文件实现动态目标管理：

[
  {"targets": ["192.168.1.10:8080", "192.168.1.11:8080"]}
]

修改Prometheus配置：

scrape_configs:
  - job_name: 'ngxtop'
    file_sd_configs:
      - files: ['/etc/prometheus/targets.json']
        refresh_interval: 30s  # 自动刷新目标列表

4. ngxtop指标暴露配置

4.1 核心指标采集逻辑

# ngxtop/ngxtop.py 第200-250行
class SQLProcessor(object):
    def __init__(self, report_queries, fields):
        self.conn = sqlite3.connect(':memory:')  # 内存数据库存储指标
        self.init_db(fields)  # 初始化指标表结构
        
    def process(self, records):
        # 插入解析后的Nginx日志记录
        insert = 'insert into log (%s) values (%s)' % (self.column_list, self.holder_list)
        for r in records:
            cursor.execute(insert, r)
            
    def report(self):
        # 生成并返回指标报告
        return self.conn.cursor().execute(query).fetchall()

4.2 自定义监控指标

通过--a参数添加自定义聚合指标：

ngxtop --a "sum(request_time)" --a "avg(bytes_sent)"

5. Grafana变量配置详解

5.1 变量定义界面

5.2 目标主机变量配置

1. 变量设置（Settings）

   • Name: target
   • Type: Query
   • Data source: Prometheus
   • Refresh: On Dashboard Load

2. 查询配置（Query Options）

   label_values(up{job="ngxtop"}, instance)  # 获取所有活跃ngxtop实例
   

3. 高级选项（Advanced）

   • Regex: /([^:]+):(\d+)/ # 提取主机名与端口
   • Sort: Disabled

6. 动态仪表盘制作

6.1 利用变量的查询示例

# CPU使用率面板查询
avg(rate(ngxtop_cpu_usage{instance=~"$target"}[5m])) * 100

6.2 完整仪表盘JSON结构

{
  "annotations": {
    "list": []
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 1,
  "iteration": 1620000000000,
  "links": [],
  "panels": [
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "Prometheus",
      "fieldConfig": {
        "defaults": {},
        "overrides": []
      },
      "fill": 1,
      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 2,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "null",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "7.5.5",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "sum(rate(ngxtop_requests_total{instance=~\"$target\"}[5m]))",
          "interval": "",
          "legendFormat": "RPS",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeRegions": [],
      "timeShift": null,
      "title": "请求速率 (RPS)",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": "0",
          "show": true
        },
        {
          "format": "short",
          "label": null,
          "logBase": 1,
          "max": null,
          "min": null,
          "show": true
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    }
  ],
  "refresh": "5s",
  "schemaVersion": 27,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": [
      {
        "allValue": null,
        "current": {
          "selected": false,
          "text": "All",
          "value": "$__all"
        },
        "datasource": "Prometheus",
        "definition": "label_values(up{job=\"ngxtop\"}, instance)",
        "description": null,
        "error": null,
        "hide": 0,
        "includeAll": true,
        "label": "Target",
        "multi": false,
        "name": "target",
        "options": [],
        "query": {
          "query": "label_values(up{job=\"ngxtop\"}, instance)",
          "refId": "StandardVariableQuery"
        },
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 1,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      }
    ]
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ]
  },
  "timezone": "",
  "title": "ngxtop动态监控",
  "uid": "ngxtop-dynamic",
  "version": 1
}

7. 最佳实践与优化

7.1 变量级联配置

实现"环境→应用→实例"三级切换：

7.2 性能优化建议

1. 变量刷新策略：

   • 低频变动变量：On Dashboard Load
   • 高频变动变量：On Time Range Change

2. 查询优化：

   • 使用label_values而非query_result
   • 添加job标签过滤减少查询范围

8. 常见问题排查

8.1 变量无数据问题

1. 检查Prometheus目标状态：up{job="ngxtop"}
2. 验证Prometheus数据源连接
3. 确认正则表达式正确性

8.2 仪表盘加载缓慢

1. 减少变量数量
2. 增大变量刷新间隔
3. 优化PromQL查询（添加适当过滤条件）

9. 总结与展望

通过ngxtop、Prometheus服务发现与Grafana变量的组合，我们构建了一套灵活高效的动态监控方案。该方案已在生产环境验证，可支持50+Nginx实例的秒级切换监控。未来可扩展至：

• 基于Consul的服务网格监控
• 结合机器学习的异常检测
• 多维度变量组合分析

建议收藏本文，并关注项目README.rst获取最新更新。下期预告：《ngxtop指标告警最佳实践》。

【免费下载链接】ngxtop Real-time metrics for nginx server 项目地址: https://gitcode.com/gh_mirrors/ng/ngxtop