自定义metrics及可视化-demo

工作中经常需要自定义metrics监控，用于发现及排查问题，而prometheus是现在最普遍用到的监控工具

本文中讲通过一个实例demo来演示创建过程

1.创建springboot项目

pom.xml 增加prometheus相关的依赖

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-core</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
        </dependency>

application.yml增加配置

management:
  endpoints:
    web:
      base-path: /monitor
      exposure:
        #加载所有的端点，默认只加载了info、health
        include: '*'
  endpoint:
    health:
      show-details: always
    #可以关闭指定的端点
    shutdown:
      enabled: true

2、自定义metrics类，实现MeterBinder接口

package com.mp.metrics;

import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.binder.MeterBinder;
import org.springframework.stereotype.Component;

import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 * ClassName: YarnMetrics
 * Function:  TODO
 * Date:      2020-01-08 08:28
 * author     mapeng
 * version    V1.0
 */
@Component
public class YarnMetrics implements MeterBinder {
    public Map<String,Double> map = new HashMap<>();

    //定义metrics name
    public final List<String> YARN_METRICS_NAMES = Arrays.asList(
            "appsCompleted",  //已完成任务数
            "appsPending",  //排队任务数
            "appsRunning", //运行job数
            "totalMB",  // 总内存大小
            "allocatedMB", // 已使用内存
            "availableMB", // 空闲内存
            "totalVirtualCores", // 总核数
            "allocatedVirtualCores", // 已用核数
            "availableVirtualCores", // 空闲核数
            "totalNodes", // 总节点数
            "activeNodes", // 激活节点数
            "decommissionedNodes" // 退役节点数
    );
    public final String YARN_MERTICS_PREFIX = "yarn_";


    @Override
    public void bindTo(MeterRegistry registry) {
        for(String s:YARN_METRICS_NAMES){
            Gauge.builder(YARN_MERTICS_PREFIX.concat(s),map, m->m.get(s))
                    .register(registry);
        }
    }
}

3、业务实现，用于填充metrics values

本demo实现的是，通过调用yarn的api接口，获取集群资源使用情况，并填充到自定义指标

package com.mp.service.impl;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.alibaba.fastjson.JSONPath;
import com.jayway.jsonpath.JsonPath;
import com.mp.metrics.YarnMetrics;
import com.mp.service.YarnService;
import com.mp.utils.RestfulUtil;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.Map;


/**
 * ClassName: YarnServiceImpl
 * Function:  TODO
 * Date:      2020-01-08 08:38
 * author     mapeng
 * version    V1.0
 */
@Service
public class YarnServiceImpl implements YarnService {
    @Autowired
    RestfulUtil restfulUtil;
    @Autowired
    YarnMetrics yarnMetrics;

    @Override
    public Map<String,Double> getYarnMetrics(String url) {
        String response = restfulUtil.getRequest(url);
        String clusterMetrics = JSON.parseObject(response).getString("clusterMetrics");
        if(StringUtils.isNotBlank(clusterMetrics)){
            JSONObject json = JSON.parseObject(clusterMetrics);
            yarnMetrics.YARN_METRICS_NAMES.forEach(metrics -> {
                double metricsValue = json.getDoubleValue(metrics);
                yarnMetrics.map.put(metrics,metricsValue);  // 设置 metrics values
            });
        }

        return yarnMetrics.map;
    }



}

4.上报prometheus

修改yml配置文件，增加上报应用targets;

- job_name: 'dw_monitor'
    metrics_path: '/monitor/prometheus'
    static_configs:
    - targets: ['localhost:28096']

注： 这里的metrics_path是springboot配置文件(application.yml)中，指定的manager.endpoints.web.base-path=/monitor 来指定的；

5.启动springboot应用

可以检查相应的端口服务

6.重启prometheus服务

7.grafana可视化展现